[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:08:46 dist-test-slave-j2vl krb5kdc[8770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:08:46 dist-test-slave-j2vl krb5kdc[8770](info): set up 2 sockets
Aug 09 23:08:46 dist-test-slave-j2vl krb5kdc[8770](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 23:08:47.545222 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:08:47 dist-test-slave-j2vl krb5kdc[8770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:08:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:08:47.681931 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44007
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:44007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:47.772408 8785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:47.772680 8785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:47.772722 8785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:47.773995 8785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:08:47.774070 8785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:47.774093 8785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:47.774117 8785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:08:47.774144 8785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:08:47.775902 8785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:44007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:47.776321 8785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:47.776605 8785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:47.783910 8797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:47.783936 8806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:47.783914 8798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:47.784783 8785 server_base.cc:1034] running on GCE node
I20260809 23:08:47.785264 8785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:47.785840 8785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:47.787036 8785 hybrid_clock.cc:648] HybridClock initialized: now 1786316927786986 us; error 64 us; skew 500 ppm
Aug 09 23:08:47 dist-test-slave-j2vl krb5kdc[8770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316927, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:47.792757 8785 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:08:47.793641 8785 webserver.cc:460] Webserver started at http://127.8.141.254:42467/ using document root <none> and password file <none>
I20260809 23:08:47.793963 8785 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:47.794042 8785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:47.794224 8785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:47.795521 8785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "c9080af0e2e94a45b99746494ae4bc1b"
format_stamp: "Formatted at 2026-08-09 23:08:47 on dist-test-slave-j2vl"
server_key: "96539494daff36ab4fa7e33322d1ee09"
server_key_iv: "605891fe379ad9e94154c6d2c72074a3"
server_key_version: "encryptionkey@0"
I20260809 23:08:47.795910 8785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "c9080af0e2e94a45b99746494ae4bc1b"
format_stamp: "Formatted at 2026-08-09 23:08:47 on dist-test-slave-j2vl"
server_key: "96539494daff36ab4fa7e33322d1ee09"
server_key_iv: "605891fe379ad9e94154c6d2c72074a3"
server_key_version: "encryptionkey@0"
I20260809 23:08:47.797950 8785 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20260809 23:08:47.799108 8815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:47.799471 8785 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:47.799594 8785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "c9080af0e2e94a45b99746494ae4bc1b"
format_stamp: "Formatted at 2026-08-09 23:08:47 on dist-test-slave-j2vl"
server_key: "96539494daff36ab4fa7e33322d1ee09"
server_key_iv: "605891fe379ad9e94154c6d2c72074a3"
server_key_version: "encryptionkey@0"
I20260809 23:08:47.799677 8785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:47.811128 8785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:47.814179 8785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:47.814404 8785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:47.820334 8785 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:44007
I20260809 23:08:47.820425 8909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:44007 every 8 connection(s)
I20260809 23:08:47.820950 8785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:08:47.822609 8910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:47.826196 8910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b: Bootstrap starting.
I20260809 23:08:47.827188 8910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:47.827550 8910 log.cc:824] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:47.828497 8910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b: No bootstrap required, opened a new log
I20260809 23:08:47.829730 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 8785
I20260809 23:08:47.829855 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:08:47.830682 8759 external_mini_cluster.cc:1428] Setting key bc79bebef0d51c81658dc91908fbc423
I20260809 23:08:47.831315 8910 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9080af0e2e94a45b99746494ae4bc1b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44007 } }
I20260809 23:08:47.831597 8910 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:47.831686 8910 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9080af0e2e94a45b99746494ae4bc1b, State: Initialized, Role: FOLLOWER
I20260809 23:08:47.831913 8910 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9080af0e2e94a45b99746494ae4bc1b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44007 } }
I20260809 23:08:47.832053 8910 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:08:47.832082 8910 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:08:47.832134 8910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:47.833406 8910 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9080af0e2e94a45b99746494ae4bc1b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44007 } }
I20260809 23:08:47.833618 8910 leader_election.cc:302] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9080af0e2e94a45b99746494ae4bc1b; no voters:
I20260809 23:08:47.833863 8910 leader_election.cc:288] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:08:47.833887 8916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:47.834012 8916 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [term 1 LEADER]: Becoming Leader. State: Replica: c9080af0e2e94a45b99746494ae4bc1b, State: Running, Role: LEADER
I20260809 23:08:47.834160 8916 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9080af0e2e94a45b99746494ae4bc1b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44007 } }
I20260809 23:08:47.834329 8910 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:08:47.834745 8919 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9080af0e2e94a45b99746494ae4bc1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9080af0e2e94a45b99746494ae4bc1b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44007 } } }
I20260809 23:08:47.835037 8919 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [sys.catalog]: This master's current role is: LEADER
I20260809 23:08:47.834934 8921 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9080af0e2e94a45b99746494ae4bc1b. Latest consensus state: current_term: 1 leader_uuid: "c9080af0e2e94a45b99746494ae4bc1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9080af0e2e94a45b99746494ae4bc1b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44007 } } }
I20260809 23:08:47.835457 8921 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b [sys.catalog]: This master's current role is: LEADER
I20260809 23:08:47.836006 8935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:08:47.837205 8935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:08:47.839962 8935 catalog_manager.cc:1347] Generated new cluster ID: 03c44855f1dc442a9c9f04770e89b9b1
I20260809 23:08:47.840018 8935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:08:47 dist-test-slave-j2vl krb5kdc[8770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:47.851962 8935 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:08:47.852643 8935 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:08:47.853482 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:47.832301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33338 (local address 127.8.141.254:44007)
0809 23:08:47.832691 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:47.832702 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:47.832933 (+ 231us) server_negotiation.cc:407] Connection header received
0809 23:08:47.835298 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:47.835337 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:47.835453 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:47.835577 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:47.838696 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:47.840336 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:47.842356 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:47.843070 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:47.843207 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:47.848329 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:47.848334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:47.848334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:47.848348 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:47.850760 (+ 2412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:47.851320 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:47.851324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:47.851324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:47.851453 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:47.851880 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:47.851881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:47.851881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:47.852187 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:47.852355 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:47.852729 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 23:08:47.852950 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":134,"threads_started":1}
I20260809 23:08:47.864383 8935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9080af0e2e94a45b99746494ae4bc1b: Generated new TSK 0
I20260809 23:08:47.864590 8935 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:08:47.901628 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44007
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:47.992234 8966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:47.992477 8966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:47.992517 8966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:47.993700 8966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:47.993765 8966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:47.993811 8966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:08:47.995534 8966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44007
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:47.995867 8966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:47.996109 8966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:48.001770 8974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:48.001960 8966 server_base.cc:1034] running on GCE node
W20260809 23:08:48.002149 8982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:48.001883 8973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:48.002466 8966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:48.002741 8966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:48.003934 8966 hybrid_clock.cc:648] HybridClock initialized: now 1786316928003916 us; error 31 us; skew 500 ppm
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:48.007975 8966 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:08:48.008641 8966 webserver.cc:460] Webserver started at http://127.8.141.193:44799/ using document root <none> and password file <none>
I20260809 23:08:48.008930 8966 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:48.009006 8966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:48.009172 8966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:48.010438 8966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "676268092bc844fc823ed350fe8e579f"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "5ca5956eca979370357bd5c28c2a3538"
server_key_iv: "b4ad2b528aaf45540932dfef9f31f7c2"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.010860 8966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "676268092bc844fc823ed350fe8e579f"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "5ca5956eca979370357bd5c28c2a3538"
server_key_iv: "b4ad2b528aaf45540932dfef9f31f7c2"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.012506 8966 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:08:48.013437 8991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.013651 8966 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:48.013793 8966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "676268092bc844fc823ed350fe8e579f"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "5ca5956eca979370357bd5c28c2a3538"
server_key_iv: "b4ad2b528aaf45540932dfef9f31f7c2"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.013880 8966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:48.037860 8966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:48.040339 8966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:48.040648 8966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:48.041083 8966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:08:48.041553 8966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:48.041636 8966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.041680 8966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:48.041721 8966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.052802 8966 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:33807
I20260809 23:08:48.052856 9196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:33807 every 8 connection(s)
I20260809 23:08:48.053226 8966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:48.060822 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 8966
I20260809 23:08:48.060926 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:08:48.061121 8759 external_mini_cluster.cc:1428] Setting key 768fbf44e0bdb95a1f51ffe8a6001f12
I20260809 23:08:48.063900 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.054170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:54653 (local address 127.8.141.254:44007)
0809 23:08:48.054234 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:48.054236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:48.054459 (+ 223us) server_negotiation.cc:407] Connection header received
0809 23:08:48.055005 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:48.055006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:48.055019 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:48.055087 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:48.056235 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.056762 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.057798 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.058027 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.058087 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.060368 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:48.060370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:48.060370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:48.060384 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:48.062323 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.062777 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.062778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.062779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.062855 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.063142 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.063142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.063143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.063316 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:48.063391 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:48.063639 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 23:08:48.063750 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:08:48.064345 9203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.054128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44007 (local address 127.8.141.193:54653)
0809 23:08:48.054311 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:48.054355 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:48.054886 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:48.055161 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:48.055164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:48.055926 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:48.056140 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:48.056144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.056841 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.056842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:48.057711 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:48.057714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.058102 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.058103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:48.058208 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.058762 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:48.058770 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:48.060257 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:48.062440 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:48.062449 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:48.062451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:48.062687 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:48.062939 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:48.062940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:48.062940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:48.063070 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:48.063386 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:48.063391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:48.063509 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 23:08:48.063596 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:08:48.063765 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
I20260809 23:08:48.064963 9198 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44007
I20260809 23:08:48.065081 9198 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:48.065304 9198 heartbeater.cc:511] Master 127.8.141.254:44007 requested a full tablet report, sending...
I20260809 23:08:48.066345 8845 ts_manager.cc:194] Registered new tserver with Master: 676268092bc844fc823ed350fe8e579f (127.8.141.193:33807)
I20260809 23:08:48.067433 8845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:54653
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:08:48.090340 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44007
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:48.185225 9206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:48.185472 9206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:48.185499 9206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:48.187296 9206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:48.187381 9206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:48.187430 9206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:08:48.189508 9206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44007
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:48.189815 9206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:48.190080 9206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:48.195705 9219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:48.195726 9215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:48.195966 9206 server_base.cc:1034] running on GCE node
W20260809 23:08:48.196080 9214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:48.196314 9206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:48.196586 9206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:48.197722 9206 hybrid_clock.cc:648] HybridClock initialized: now 1786316928197698 us; error 42 us; skew 500 ppm
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:48.200361 9206 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:08:48.200805 9206 webserver.cc:460] Webserver started at http://127.8.141.194:35871/ using document root <none> and password file <none>
I20260809 23:08:48.201052 9206 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:48.201098 9206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:48.201263 9206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:48.202432 9206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "55f63149e20541858e36c35ce91c9328"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "9085b15df1ab3e412ff463b5e2db1bce"
server_key_iv: "a6a11cd0f182c9000e9aee89c3829497"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.202801 9206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "55f63149e20541858e36c35ce91c9328"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "9085b15df1ab3e412ff463b5e2db1bce"
server_key_iv: "a6a11cd0f182c9000e9aee89c3829497"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.204262 9206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:08:48.205096 9238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.205291 9206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:48.205425 9206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "55f63149e20541858e36c35ce91c9328"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "9085b15df1ab3e412ff463b5e2db1bce"
server_key_iv: "a6a11cd0f182c9000e9aee89c3829497"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.205488 9206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:48.233129 9206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:48.236264 9206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:48.236424 9206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:48.236649 9206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:08:48.236945 9206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:48.236979 9206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.237004 9206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:48.237017 9206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.245970 9206 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:39821
I20260809 23:08:48.246037 9431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:39821 every 8 connection(s)
I20260809 23:08:48.246387 9206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:08:48.247195 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9206
I20260809 23:08:48.247334 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:08:48.247538 8759 external_mini_cluster.cc:1428] Setting key baaf9b77db81146b05de499fc8f131e4
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
I20260809 23:08:48.257880 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.247204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:60553 (local address 127.8.141.254:44007)
0809 23:08:48.247267 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:48.247269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:48.247657 (+ 388us) server_negotiation.cc:407] Connection header received
0809 23:08:48.248382 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:48.248383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:48.248393 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:48.248434 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:48.250033 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.250544 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.251664 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.251874 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.251960 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.254732 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:48.254735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:48.254735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:48.254749 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:48.256329 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.256802 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.256804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.256804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.256866 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.257159 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.257160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.257160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.257286 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:48.257386 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:48.257684 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 23:08:48.257779 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:08:48.258100 9438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.247202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44007 (local address 127.8.141.194:60553)
0809 23:08:48.247491 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:48.247538 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:48.248258 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:48.248512 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:48.248516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:48.249569 (+ 1053us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:48.249914 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:48.249922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.250685 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.250686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:48.251563 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:48.251566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.251978 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.251979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:48.252124 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.252781 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:48.252792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:48.254604 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:48.256453 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:48.256462 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:48.256465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:48.256718 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:48.256951 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:48.256952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:48.256953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:48.257078 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:48.257369 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:48.257377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:48.257539 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 23:08:48.257647 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 23:08:48.257839 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":94,"threads_started":1}
I20260809 23:08:48.258682 9432 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44007
I20260809 23:08:48.258787 9432 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:48.258949 9432 heartbeater.cc:511] Master 127.8.141.254:44007 requested a full tablet report, sending...
I20260809 23:08:48.259311 8845 ts_manager.cc:194] Registered new tserver with Master: 55f63149e20541858e36c35ce91c9328 (127.8.141.194:39821)
I20260809 23:08:48.259683 8845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:60553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:08:48.280373 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44007
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:44037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:48.376197 9444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:48.376470 9444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:48.376513 9444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:48.377739 9444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:48.377811 9444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:48.377859 9444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:08:48.379453 9444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:44037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44007
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:48.379755 9444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:48.380061 9444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:48.385913 9452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:48.385926 9453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:48.385924 9455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:48.386202 9444 server_base.cc:1034] running on GCE node
I20260809 23:08:48.386449 9444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:48.386672 9444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:48.387854 9444 hybrid_clock.cc:648] HybridClock initialized: now 1786316928387628 us; error 246 us; skew 500 ppm
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:48.390930 9444 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:08:48.391453 9444 webserver.cc:460] Webserver started at http://127.8.141.195:39335/ using document root <none> and password file <none>
I20260809 23:08:48.391701 9444 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:48.391777 9444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:48.391981 9444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:48.393087 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "11d1cbf54a314a169eb56520012f0028"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "0187428ee46a7e189a1b8637e10f8115"
server_key_iv: "a193656856dfcc4ebdbd96fa12585e8e"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.393523 9444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "11d1cbf54a314a169eb56520012f0028"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "0187428ee46a7e189a1b8637e10f8115"
server_key_iv: "a193656856dfcc4ebdbd96fa12585e8e"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.395126 9444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:48.396027 9466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.396255 9444 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:48.396378 9444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "11d1cbf54a314a169eb56520012f0028"
format_stamp: "Formatted at 2026-08-09 23:08:48 on dist-test-slave-j2vl"
server_key: "0187428ee46a7e189a1b8637e10f8115"
server_key_iv: "a193656856dfcc4ebdbd96fa12585e8e"
server_key_version: "encryptionkey@0"
I20260809 23:08:48.396468 9444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:48.410434 9444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:48.413101 9444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:48.413307 9444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:48.413715 9444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:08:48.414104 9444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:48.414160 9444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.414198 9444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:48.414239 9444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:48.423296 9444 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:46777
I20260809 23:08:48.423348 9688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:46777 every 8 connection(s)
I20260809 23:08:48.423727 9444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:08:48.427561 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9444
I20260809 23:08:48.427662 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:08:48.427821 8759 external_mini_cluster.cc:1428] Setting key 2bad68a4ce405432b031ac1dcb25ab3f
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:48.434592 8914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.424589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:57735 (local address 127.8.141.254:44007)
0809 23:08:48.424659 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:48.424661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:48.424949 (+ 288us) server_negotiation.cc:407] Connection header received
0809 23:08:48.425501 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:48.425502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:48.425514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:48.425583 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:48.426767 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.427375 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.428901 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.429186 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.429283 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.431687 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:48.431688 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:48.431688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:48.431696 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:48.433092 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.433541 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.433542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.433542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.433610 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.433969 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.433971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.433971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.434095 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:48.434155 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:48.434451 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 23:08:48.434522 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 23:08:48.434803 9692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.424561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44007 (local address 127.8.141.195:57735)
0809 23:08:48.424766 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:48.424822 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:48.425392 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:48.425661 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:48.425664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:48.426362 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:48.426647 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:48.426652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.427501 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.427503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:48.428815 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:48.428817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.429307 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.429308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:48.429500 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.430027 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:48.430037 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:48.431586 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:48.433169 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:48.433175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:48.433177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:48.433462 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:48.433701 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:48.433701 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:48.433702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:48.433804 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:48.434180 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:48.434188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:48.434323 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 23:08:48.434415 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:08:48.434562 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":113,"threads_started":1}
I20260809 23:08:48.435355 9689 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44007
I20260809 23:08:48.435468 9689 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:48.435657 9689 heartbeater.cc:511] Master 127.8.141.254:44007 requested a full tablet report, sending...
I20260809 23:08:48.436132 8845 ts_manager.cc:194] Registered new tserver with Master: 11d1cbf54a314a169eb56520012f0028 (127.8.141.195:46777)
I20260809 23:08:48.436549 8845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:57735
I20260809 23:08:48.440268 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:08:48.468936 9702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.458797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47018 (local address 127.8.141.193:33807)
0809 23:08:48.459072 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:48.459078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:48.459091 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:08:48.459180 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:48.459185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:48.459207 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:48.459303 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:48.459921 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.460674 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.461650 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.462160 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.462261 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.465245 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:48.465257 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:48.465257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:48.465265 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:48.467462 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.467900 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.467904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.467904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.467976 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.468178 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.468180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.468180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.468392 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:48.468555 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:48.468656 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 23:08:48.468755 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":125,"threads_started":1}
W20260809 23:08:48.469513 9044 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47018
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[8770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316928, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:08:48.496562 9702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:48.488454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47028 (local address 127.8.141.193:33807)
0809 23:08:48.488517 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:48.488518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:48.488587 (+ 69us) server_negotiation.cc:407] Connection header received
0809 23:08:48.488645 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:48.488646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:48.488658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:48.488723 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:48.489452 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.489998 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.490805 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:48.491042 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:48.491111 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:48.493757 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:48.493759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:48.493759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:48.493768 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:48.495419 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.495798 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.495800 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.495800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.495861 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:48.496093 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:48.496094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:48.496094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:48.496199 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:48.496385 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:48.496417 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 23:08:48.496490 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:08:48.497234 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 8966
I20260809 23:08:48.514442 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9206
I20260809 23:08:48.524127 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9444
I20260809 23:08:48.537065 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 8785
2026-08-09T23:08:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2384 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[9711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[9711](info): set up 2 sockets
Aug 09 23:08:48 dist-test-slave-j2vl krb5kdc[9711](info): commencing operation
krb5kdc: starting...
W20260809 23:08:49.815984 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:08:49 dist-test-slave-j2vl krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:08:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:08:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:08:49.953509 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:35459
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:35459
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:50.041008 9736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:50.041285 9736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:50.041330 9736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:50.042537 9736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:08:50.042615 9736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:50.042640 9736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:50.042663 9736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:08:50.042692 9736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:08:50.044236 9736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:35459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:35459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:50.044531 9736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:50.044787 9736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:50.049947 9749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:50.049962 9747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:50.050199 9736 server_base.cc:1034] running on GCE node
W20260809 23:08:50.049988 9752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:50.050556 9736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:50.050885 9736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:50.052043 9736 hybrid_clock.cc:648] HybridClock initialized: now 1786316930052027 us; error 31 us; skew 500 ppm
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:50.054872 9736 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:08:50.055361 9736 webserver.cc:460] Webserver started at http://127.8.141.254:41043/ using document root <none> and password file <none>
I20260809 23:08:50.055608 9736 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:50.055658 9736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:50.055790 9736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:50.056915 9736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "88ab3e543f6f429d846be9eaf2c706fa"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "e1283d75af16a3e0e5c0f1e7b03b84f3"
server_key_iv: "4cb5458cdd129307f9b837c864681acb"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.057305 9736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "88ab3e543f6f429d846be9eaf2c706fa"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "e1283d75af16a3e0e5c0f1e7b03b84f3"
server_key_iv: "4cb5458cdd129307f9b837c864681acb"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.058921 9736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:50.059883 9762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.060105 9736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:50.060206 9736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "88ab3e543f6f429d846be9eaf2c706fa"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "e1283d75af16a3e0e5c0f1e7b03b84f3"
server_key_iv: "4cb5458cdd129307f9b837c864681acb"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.060271 9736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:50.076841 9736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:50.079301 9736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:50.079435 9736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:50.084182 9736 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:35459
I20260809 23:08:50.084251 9854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:35459 every 8 connection(s)
I20260809 23:08:50.084637 9736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:08:50.085786 9855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:50.088178 9855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa: Bootstrap starting.
I20260809 23:08:50.088765 9855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:50.089020 9855 log.cc:824] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:50.089303 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9736
I20260809 23:08:50.089468 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:08:50.089653 8759 external_mini_cluster.cc:1428] Setting key cb02175f853c89cacfeadbcd9a11aed9
I20260809 23:08:50.090025 9855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa: No bootstrap required, opened a new log
I20260809 23:08:50.091784 9855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ab3e543f6f429d846be9eaf2c706fa" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 35459 } }
I20260809 23:08:50.091924 9855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:50.091948 9855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88ab3e543f6f429d846be9eaf2c706fa, State: Initialized, Role: FOLLOWER
I20260809 23:08:50.092109 9855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ab3e543f6f429d846be9eaf2c706fa" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 35459 } }
I20260809 23:08:50.092216 9855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:08:50.092264 9855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:08:50.092336 9855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:50.093222 9855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ab3e543f6f429d846be9eaf2c706fa" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 35459 } }
I20260809 23:08:50.093349 9855 leader_election.cc:302] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88ab3e543f6f429d846be9eaf2c706fa; no voters:
I20260809 23:08:50.093587 9855 leader_election.cc:288] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:08:50.093622 9861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:50.093741 9861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [term 1 LEADER]: Becoming Leader. State: Replica: 88ab3e543f6f429d846be9eaf2c706fa, State: Running, Role: LEADER
I20260809 23:08:50.093868 9855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:08:50.093889 9861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ab3e543f6f429d846be9eaf2c706fa" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 35459 } }
I20260809 23:08:50.094327 9861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88ab3e543f6f429d846be9eaf2c706fa. Latest consensus state: current_term: 1 leader_uuid: "88ab3e543f6f429d846be9eaf2c706fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ab3e543f6f429d846be9eaf2c706fa" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 35459 } } }
I20260809 23:08:50.094491 9861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [sys.catalog]: This master's current role is: LEADER
I20260809 23:08:50.094789 9878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:08:50.095369 9878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:08:50.096375 9865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88ab3e543f6f429d846be9eaf2c706fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88ab3e543f6f429d846be9eaf2c706fa" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 35459 } } }
I20260809 23:08:50.096462 9865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa [sys.catalog]: This master's current role is: LEADER
I20260809 23:08:50.097833 9878 catalog_manager.cc:1347] Generated new cluster ID: 27b8eda80b354d3a818b119fafb564f3
I20260809 23:08:50.097891 9878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:50.102566 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.090210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58340 (local address 127.8.141.254:35459)
0809 23:08:50.090409 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.090419 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.090458 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:08:50.091795 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.091805 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.091869 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.092063 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.092824 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.094019 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.095076 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.095681 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.095799 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.098690 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.098694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.098695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.098707 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.100429 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.100814 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.100817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.100818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.100941 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.101166 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.101167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.101167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.101487 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.102246 (+ 759us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.102307 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.102418 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260809 23:08:50.110056 9878 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:08:50.110801 9878 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:08:50.117285 9878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88ab3e543f6f429d846be9eaf2c706fa: Generated new TSK 0
I20260809 23:08:50.117544 9878 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:08:50.152752 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:35459
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:50.268596 9900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:50.268864 9900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:50.268911 9900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:50.270232 9900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:50.270306 9900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:50.270357 9900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:08:50.271982 9900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:35459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:50.272279 9900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:50.272529 9900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:50.277933 9909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:50.277923 9910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:50.278384 9913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:50.278475 9900 server_base.cc:1034] running on GCE node
I20260809 23:08:50.278718 9900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:50.278990 9900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:50.280174 9900 hybrid_clock.cc:648] HybridClock initialized: now 1786316930280154 us; error 38 us; skew 500 ppm
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:50.283017 9900 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:08:50.283540 9900 webserver.cc:460] Webserver started at http://127.8.141.193:40585/ using document root <none> and password file <none>
I20260809 23:08:50.283790 9900 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:50.283864 9900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:50.284024 9900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:50.285113 9900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "3e4153eb093f4a86b413000194a22a30"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "f227b69e49d1b04582788949155c2a9b"
server_key_iv: "07dea366625450aad26b8a27091537a8"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.285521 9900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "3e4153eb093f4a86b413000194a22a30"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "f227b69e49d1b04582788949155c2a9b"
server_key_iv: "07dea366625450aad26b8a27091537a8"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.286928 9900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:50.287997 9928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.288278 9900 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:50.288414 9900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "3e4153eb093f4a86b413000194a22a30"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "f227b69e49d1b04582788949155c2a9b"
server_key_iv: "07dea366625450aad26b8a27091537a8"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.288496 9900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:50.304390 9900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:50.306721 9900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:50.306912 9900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:50.307191 9900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:08:50.307535 9900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:50.307600 9900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.307637 9900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:50.307683 9900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.314293 9900 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:42659
I20260809 23:08:50.314352 10132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:42659 every 8 connection(s)
I20260809 23:08:50.314679 9900 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:08:50.319815 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9900
I20260809 23:08:50.319945 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:08:50.320122 8759 external_mini_cluster.cc:1428] Setting key d80d9cb463fb9a6fa852a3633f7600b1
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:50.326804 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.315547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:51665 (local address 127.8.141.254:35459)
0809 23:08:50.315651 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.315653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.315890 (+ 237us) server_negotiation.cc:407] Connection header received
0809 23:08:50.317242 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.317243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.317262 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.317310 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.318092 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.318670 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.319832 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.320141 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.320225 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.323128 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.323130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.323130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.323141 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.324853 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.325631 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.325633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.325633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.325712 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.326067 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.326067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.326068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.326195 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.326251 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.326646 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.326734 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 23:08:50.327131 10136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.315422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:35459 (local address 127.8.141.193:51665)
0809 23:08:50.315707 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.315769 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.317108 (+ 1339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.317432 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.317437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.317745 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:50.317961 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.317965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.318787 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.318788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.319709 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.319711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.320238 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.320239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.320402 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.321129 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:50.321141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:50.323018 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:50.324958 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:50.324968 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:50.324970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:50.325496 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:50.325801 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:50.325802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:50.325802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:50.325967 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:50.326268 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:50.326276 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:50.326472 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.326601 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.326821 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":109,"threads_started":1}
I20260809 23:08:50.327874 10133 heartbeater.cc:347] Connected to a master server at 127.8.141.254:35459
I20260809 23:08:50.328001 10133 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:50.328212 10133 heartbeater.cc:511] Master 127.8.141.254:35459 requested a full tablet report, sending...
I20260809 23:08:50.328809 9788 ts_manager.cc:194] Registered new tserver with Master: 3e4153eb093f4a86b413000194a22a30 (127.8.141.193:42659)
I20260809 23:08:50.329537 9788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:51665
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:08:50.353444 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:35459
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:50.443246 10142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:50.443475 10142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:50.443502 10142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:50.444728 10142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:50.444777 10142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:50.444806 10142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:08:50.446527 10142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:35459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:50.446843 10142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:50.447152 10142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:50.452795 10151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:50.452872 10149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:50.452795 10156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:50.453023 10142 server_base.cc:1034] running on GCE node
I20260809 23:08:50.453296 10142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:50.453598 10142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:50.454779 10142 hybrid_clock.cc:648] HybridClock initialized: now 1786316930454764 us; error 31 us; skew 500 ppm
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:50.457715 10142 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:08:50.458220 10142 webserver.cc:460] Webserver started at http://127.8.141.194:41699/ using document root <none> and password file <none>
I20260809 23:08:50.458480 10142 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:50.458527 10142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:50.458663 10142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:50.459792 10142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "181cbfe07d404b4b85c1f467f9532d72"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "b4a50472e3d1805457799af098dfff81"
server_key_iv: "f61b7e1c796e0c7bfd0fe5e80edd262f"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.460173 10142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "181cbfe07d404b4b85c1f467f9532d72"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "b4a50472e3d1805457799af098dfff81"
server_key_iv: "f61b7e1c796e0c7bfd0fe5e80edd262f"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.461751 10142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:08:50.463228 10166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.463528 10142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:50.463701 10142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "181cbfe07d404b4b85c1f467f9532d72"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "b4a50472e3d1805457799af098dfff81"
server_key_iv: "f61b7e1c796e0c7bfd0fe5e80edd262f"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.463872 10142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:50.495713 10142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:50.498745 10142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:50.498915 10142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:50.499208 10142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:08:50.499562 10142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:50.499619 10142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.499663 10142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:50.499691 10142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.506331 10142 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:33723
I20260809 23:08:50.506388 10339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:33723 every 8 connection(s)
I20260809 23:08:50.506722 10142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:08:50.510058 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10142
I20260809 23:08:50.510141 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:08:50.510317 8759 external_mini_cluster.cc:1428] Setting key 9e8f2e58c9fbaa7e7d53b0dab2f5d5ab
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:50.518285 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.507467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:52803 (local address 127.8.141.254:35459)
0809 23:08:50.507556 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.507559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.507766 (+ 207us) server_negotiation.cc:407] Connection header received
0809 23:08:50.508757 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.508759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.508771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.508854 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.509607 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.510222 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.511486 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.511677 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.511750 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.514548 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.514550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.514550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.514582 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.516405 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.517014 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.517016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.517017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.517091 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.517409 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.517411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.517411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.517601 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.517686 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.518060 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.518169 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:08:50.518447 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.507433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:35459 (local address 127.8.141.194:52803)
0809 23:08:50.507634 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.507674 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.508639 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.508933 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.508936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.509208 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:50.509507 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.509512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.510318 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.510319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.511393 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.511397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.511763 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.511764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.511928 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.512467 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:50.512478 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:50.514425 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:50.516639 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:50.516648 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:50.516650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:50.516923 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:50.517173 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:50.517173 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:50.517174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:50.517294 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:50.517708 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:50.517716 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:50.517869 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.517998 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.518157 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260809 23:08:50.519135 10341 heartbeater.cc:347] Connected to a master server at 127.8.141.254:35459
I20260809 23:08:50.519282 10341 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:50.519519 10341 heartbeater.cc:511] Master 127.8.141.254:35459 requested a full tablet report, sending...
I20260809 23:08:50.519990 9788 ts_manager.cc:194] Registered new tserver with Master: 181cbfe07d404b4b85c1f467f9532d72 (127.8.141.194:33723)
I20260809 23:08:50.520569 9788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:52803
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:08:50.545926 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:35459
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:50.640550 10348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:50.640799 10348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:50.640827 10348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:50.642208 10348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:50.642266 10348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:50.642298 10348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:08:50.643996 10348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:35459
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:50.644299 10348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:50.644582 10348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:50.650990 10363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:50.651108 10357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:50.651090 10348 server_base.cc:1034] running on GCE node
W20260809 23:08:50.650997 10358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:50.651441 10348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:50.651742 10348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:50.652940 10348 hybrid_clock.cc:648] HybridClock initialized: now 1786316930652882 us; error 60 us; skew 500 ppm
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:50.656011 10348 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:08:50.656589 10348 webserver.cc:460] Webserver started at http://127.8.141.195:40823/ using document root <none> and password file <none>
I20260809 23:08:50.656880 10348 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:50.656965 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:50.657132 10348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:50.658311 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "304935fb910c4786a349000559b1b80f"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "99d66c1b738c8fde1698d825411a30e8"
server_key_iv: "0fb18151f63657f44be55a669e71fd23"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.658707 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "304935fb910c4786a349000559b1b80f"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "99d66c1b738c8fde1698d825411a30e8"
server_key_iv: "0fb18151f63657f44be55a669e71fd23"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.661111 10348 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 23:08:50.662618 10373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.662966 10348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:50.663123 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "304935fb910c4786a349000559b1b80f"
format_stamp: "Formatted at 2026-08-09 23:08:50 on dist-test-slave-j2vl"
server_key: "99d66c1b738c8fde1698d825411a30e8"
server_key_iv: "0fb18151f63657f44be55a669e71fd23"
server_key_version: "encryptionkey@0"
I20260809 23:08:50.663375 10348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:50.682791 10348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:50.685096 10348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:50.685288 10348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:50.685693 10348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:08:50.686060 10348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:50.686117 10348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.686161 10348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:50.686206 10348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:50.694448 10348 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:42713
I20260809 23:08:50.694556 10602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:42713 every 8 connection(s)
I20260809 23:08:50.694967 10348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:50.702950 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10348
I20260809 23:08:50.703076 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:08:50.703377 8759 external_mini_cluster.cc:1428] Setting key b3fc463159a6a5f43cb2f20f6b301ac2
I20260809 23:08:50.705397 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.695837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:50305 (local address 127.8.141.254:35459)
0809 23:08:50.695932 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.695934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.696139 (+ 205us) server_negotiation.cc:407] Connection header received
0809 23:08:50.697182 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.697184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.697198 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.697249 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.697974 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.698506 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.699557 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.699778 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.699855 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.702369 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.702371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.702371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.702379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.703919 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.704367 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.704368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.704369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.704429 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.704697 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.704698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.704699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.704824 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.704898 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.705188 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.705273 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 23:08:50.705631 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.695803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:35459 (local address 127.8.141.195:50305)
0809 23:08:50.696006 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.696046 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.697053 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.697334 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.697369 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.697667 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:50.697873 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.697878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.698612 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.698613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.699458 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.699461 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.699864 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.699865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.700003 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.700576 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:50.700585 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:50.702255 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:50.704009 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:50.704017 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:50.704020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:50.704280 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:50.704506 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:50.704507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:50.704507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:50.704624 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:50.704901 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:50.704909 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:50.705046 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.705152 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.705331 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260809 23:08:50.706161 10603 heartbeater.cc:347] Connected to a master server at 127.8.141.254:35459
I20260809 23:08:50.706286 10603 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:50.706489 10603 heartbeater.cc:511] Master 127.8.141.254:35459 requested a full tablet report, sending...
I20260809 23:08:50.706892 9788 ts_manager.cc:194] Registered new tserver with Master: 304935fb910c4786a349000559b1b80f (127.8.141.195:42713)
I20260809 23:08:50.707306 9788 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:50305
I20260809 23:08:50.715432 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:08:50.724921 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.717782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.8.141.254:35459)
0809 23:08:50.717857 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.717859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.717872 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:08:50.717992 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.717993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.718005 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.718068 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.718693 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.719243 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.720142 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.720358 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.720424 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.721865 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.721867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.721867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.721874 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.723387 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.723787 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.723789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.723789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.723855 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.724148 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.724149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.724149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.724359 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.724446 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.724648 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.724732 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:08:50.728821 9788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:08:50.730989 9788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:50.739655 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.735056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.8.141.194:33723)
0809 23:08:50.735253 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.735259 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.735270 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:08:50.735427 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.735431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.735458 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.735533 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:50.735937 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.736846 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.738448 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.739177 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.739259 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.739322 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.739509 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.739562 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260809 23:08:50.739751 10619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.734947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:33723 (local address 127.0.0.1:44184)
0809 23:08:50.735152 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.735197 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.735274 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.735646 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.735649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.735652 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:50.735753 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.735756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.736958 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.736959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.738367 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.738371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.739286 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.739287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.739379 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.739396 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.739474 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.739530 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":84,"threads_started":1}
I20260809 23:08:50.739969 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.735067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42713 (local address 127.0.0.1:46696)
0809 23:08:50.735786 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.735795 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.735845 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.736128 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.736129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.736131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:50.736242 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.736245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.737624 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.737625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.738679 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.738681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.739701 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.739702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.739788 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.739795 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.739854 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.739903 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":109,"threads_started":1}
I20260809 23:08:50.740330 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.735337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.8.141.195:42713)
0809 23:08:50.735524 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.735528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.735873 (+ 345us) server_negotiation.cc:407] Connection header received
0809 23:08:50.735929 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.735932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.735948 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.736032 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:50.736360 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.737506 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.738760 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.739537 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.739608 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.739663 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.740165 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.740223 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":112,"threads_started":1}
I20260809 23:08:50.740739 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.735056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:42659 (local address 127.0.0.1:58514)
0809 23:08:50.735649 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.735660 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.735709 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.736192 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.736193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.736196 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:50.736239 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.736241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.738753 (+ 2512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.738754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.739567 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.739570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.740473 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.740474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.740543 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.740548 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.740602 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.740649 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":121,"threads_started":1}
I20260809 23:08:50.741050 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.735398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58514 (local address 127.8.141.193:42659)
0809 23:08:50.735922 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.735927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.735936 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:08:50.735975 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.735978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.736004 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.736096 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:50.736330 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.737490 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.739746 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.740372 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.740469 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.740530 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.740913 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.740969 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":108,"threads_started":1}
I20260809 23:08:50.741400 10465 tablet_service.cc:1467] Processing CreateTablet for tablet f8e97d2bfb03406983a27119cca17db4 (DEFAULT_TABLE table=test-table [id=9a9d775522e04318b8f10a7ebd5d3116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:50.741590 10018 tablet_service.cc:1467] Processing CreateTablet for tablet f8e97d2bfb03406983a27119cca17db4 (DEFAULT_TABLE table=test-table [id=9a9d775522e04318b8f10a7ebd5d3116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:50.741783 10465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e97d2bfb03406983a27119cca17db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:50.741869 10018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e97d2bfb03406983a27119cca17db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:50.743494 10236 tablet_service.cc:1467] Processing CreateTablet for tablet f8e97d2bfb03406983a27119cca17db4 (DEFAULT_TABLE table=test-table [id=9a9d775522e04318b8f10a7ebd5d3116]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:50.743848 10236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e97d2bfb03406983a27119cca17db4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:50.744617 10630 tablet_bootstrap.cc:492] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f: Bootstrap starting.
I20260809 23:08:50.745180 10631 tablet_bootstrap.cc:492] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30: Bootstrap starting.
I20260809 23:08:50.745535 10630 tablet_bootstrap.cc:654] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:50.746006 10631 tablet_bootstrap.cc:654] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:50.746277 10630 log.cc:824] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:50.746591 10631 log.cc:824] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:50.747180 10630 tablet_bootstrap.cc:492] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f: No bootstrap required, opened a new log
I20260809 23:08:50.747277 10630 ts_tablet_manager.cc:1397] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:08:50.747424 10631 tablet_bootstrap.cc:492] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30: No bootstrap required, opened a new log
I20260809 23:08:50.747515 10631 ts_tablet_manager.cc:1397] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:08:50.748050 10638 tablet_bootstrap.cc:492] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72: Bootstrap starting.
I20260809 23:08:50.748732 10638 tablet_bootstrap.cc:654] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:50.749177 10630 raft_consensus.cc:348] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.749392 10638 log.cc:824] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:50.749413 10630 raft_consensus.cc:374] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:50.749609 10630 raft_consensus.cc:729] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 304935fb910c4786a349000559b1b80f, State: Initialized, Role: FOLLOWER
I20260809 23:08:50.749748 10630 consensus_queue.cc:260] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.750128 10630 ts_tablet_manager.cc:1428] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:08:50.750142 10603 heartbeater.cc:503] Master 127.8.141.254:35459 was elected leader, sending a full tablet report...
I20260809 23:08:50.750161 10631 raft_consensus.cc:348] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.750350 10631 raft_consensus.cc:374] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:50.750391 10631 raft_consensus.cc:729] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e4153eb093f4a86b413000194a22a30, State: Initialized, Role: FOLLOWER
I20260809 23:08:50.750451 10638 tablet_bootstrap.cc:492] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72: No bootstrap required, opened a new log
I20260809 23:08:50.750519 10638 ts_tablet_manager.cc:1397] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 23:08:50.750523 10631 consensus_queue.cc:260] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.750819 10631 ts_tablet_manager.cc:1428] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 23:08:50.750862 10133 heartbeater.cc:503] Master 127.8.141.254:35459 was elected leader, sending a full tablet report...
I20260809 23:08:50.752622 10638 raft_consensus.cc:348] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.752759 10638 raft_consensus.cc:374] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:50.752781 10638 raft_consensus.cc:729] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 181cbfe07d404b4b85c1f467f9532d72, State: Initialized, Role: FOLLOWER
I20260809 23:08:50.752894 10638 consensus_queue.cc:260] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.753162 10638 ts_tablet_manager.cc:1428] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:08:50.753191 10341 heartbeater.cc:503] Master 127.8.141.254:35459 was elected leader, sending a full tablet report...
W20260809 23:08:50.757295 10342 tablet.cc:2369] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:08:50.815594 10134 tablet.cc:2369] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:08:50.878490 10645 raft_consensus.cc:482] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:50.878646 10645 raft_consensus.cc:504] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.879276 10645 leader_election.cc:288] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 181cbfe07d404b4b85c1f467f9532d72 (127.8.141.194:33723), 304935fb910c4786a349000559b1b80f (127.8.141.195:42713)
I20260809 23:08:50.883738 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.879719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:43847 (local address 127.8.141.195:42713)
0809 23:08:50.879766 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.879769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.879780 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:08:50.879824 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.879825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.879835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.879910 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:50.880207 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.880981 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.882712 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.883325 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.883410 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.883419 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.883623 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.883662 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:08:50.883738 10623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.879682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:47721 (local address 127.8.141.194:33723)
0809 23:08:50.879766 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.879769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.879779 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:08:50.879814 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.879815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.879825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.879910 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:50.880204 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.880987 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.882707 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.883330 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.883410 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.883419 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.883623 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.883662 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:08:50.883782 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.879508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42713 (local address 127.8.141.193:43847)
0809 23:08:50.879647 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.879659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.879721 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.879957 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.879959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.879963 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:50.880080 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.880083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.881093 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.881094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.882600 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.882603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.883430 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.883431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.883532 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.883541 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.883605 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.883666 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":117,"thread_start_us":69,"threads_started":1}
I20260809 23:08:50.883805 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.879508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:33723 (local address 127.8.141.193:47721)
0809 23:08:50.879656 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:50.879665 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:50.879720 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:50.879957 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:50.879959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:50.879963 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:50.880082 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.880083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.881093 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.881094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.882599 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:50.882603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.883432 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.883432 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:50.883532 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.883539 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:08:50.883605 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:08:50.883666 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":172,"thread_start_us":68,"threads_started":1}
I20260809 23:08:50.884198 10509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e97d2bfb03406983a27119cca17db4" candidate_uuid: "3e4153eb093f4a86b413000194a22a30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "304935fb910c4786a349000559b1b80f" is_pre_election: true
I20260809 23:08:50.884430 10509 raft_consensus.cc:2454] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e4153eb093f4a86b413000194a22a30 in term 0.
I20260809 23:08:50.884650 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e97d2bfb03406983a27119cca17db4" candidate_uuid: "3e4153eb093f4a86b413000194a22a30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181cbfe07d404b4b85c1f467f9532d72" is_pre_election: true
I20260809 23:08:50.884764 9934 leader_election.cc:302] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 304935fb910c4786a349000559b1b80f, 3e4153eb093f4a86b413000194a22a30; no voters:
I20260809 23:08:50.884881 10265 raft_consensus.cc:2454] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e4153eb093f4a86b413000194a22a30 in term 0.
I20260809 23:08:50.884910 10645 raft_consensus.cc:2781] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:50.885012 10645 raft_consensus.cc:482] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:50.885051 10645 raft_consensus.cc:3037] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:50.885895 10645 raft_consensus.cc:504] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.886082 10645 leader_election.cc:288] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [CANDIDATE]: Term 1 election: Requested vote from peers 181cbfe07d404b4b85c1f467f9532d72 (127.8.141.194:33723), 304935fb910c4786a349000559b1b80f (127.8.141.195:42713)
I20260809 23:08:50.886278 10509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e97d2bfb03406983a27119cca17db4" candidate_uuid: "3e4153eb093f4a86b413000194a22a30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "304935fb910c4786a349000559b1b80f"
I20260809 23:08:50.886272 10265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e97d2bfb03406983a27119cca17db4" candidate_uuid: "3e4153eb093f4a86b413000194a22a30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "181cbfe07d404b4b85c1f467f9532d72"
I20260809 23:08:50.886366 10509 raft_consensus.cc:3037] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:50.886386 10265 raft_consensus.cc:3037] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:50.887351 10509 raft_consensus.cc:2454] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e4153eb093f4a86b413000194a22a30 in term 1.
I20260809 23:08:50.887497 10265 raft_consensus.cc:2454] T f8e97d2bfb03406983a27119cca17db4 P 181cbfe07d404b4b85c1f467f9532d72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e4153eb093f4a86b413000194a22a30 in term 1.
I20260809 23:08:50.887542 9934 leader_election.cc:302] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 304935fb910c4786a349000559b1b80f, 3e4153eb093f4a86b413000194a22a30; no voters:
I20260809 23:08:50.887677 10645 raft_consensus.cc:2781] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:50.887880 10645 raft_consensus.cc:686] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [term 1 LEADER]: Becoming Leader. State: Replica: 3e4153eb093f4a86b413000194a22a30, State: Running, Role: LEADER
I20260809 23:08:50.887997 10645 consensus_queue.cc:237] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } }
I20260809 23:08:50.889050 9787 catalog_manager.cc:5495] T f8e97d2bfb03406983a27119cca17db4 P 3e4153eb093f4a86b413000194a22a30 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e4153eb093f4a86b413000194a22a30 (127.8.141.193). New cstate: current_term: 1 leader_uuid: "3e4153eb093f4a86b413000194a22a30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "181cbfe07d404b4b85c1f467f9532d72" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 33723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e4153eb093f4a86b413000194a22a30" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42659 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "304935fb910c4786a349000559b1b80f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42713 } health_report { overall_health: UNKNOWN } } }
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:08:50.932544 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.924383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.8.141.193:42659)
0809 23:08:50.924505 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.924508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.924597 (+ 89us) server_negotiation.cc:407] Connection header received
0809 23:08:50.924671 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.924672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.924688 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.924750 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.925468 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.925941 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.926705 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.926900 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.926979 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.929531 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.929548 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.929548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.929560 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.931289 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.931711 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.931719 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.931720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.931779 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.931980 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.931981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.931981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.932136 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.932314 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.932354 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.932434 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260809 23:08:50.945641 10604 tablet.cc:2369] T f8e97d2bfb03406983a27119cca17db4 P 304935fb910c4786a349000559b1b80f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:08:50.959669 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.951686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58546 (local address 127.8.141.193:42659)
0809 23:08:50.951789 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.951792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.951945 (+ 153us) server_negotiation.cc:407] Connection header received
0809 23:08:50.952043 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.952045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.952058 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.952128 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.952746 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.953266 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.954077 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.954277 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.954363 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.956692 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.956694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.956694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.956703 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.958253 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.958688 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.958689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.958690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.958754 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.958977 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.958978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.958979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.959097 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.959255 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.959445 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.959547 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:08:50 dist-test-slave-j2vl krb5kdc[9711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:08:50.990041 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:50.980671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58562 (local address 127.8.141.193:42659)
0809 23:08:50.980762 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:50.980765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:50.981049 (+ 284us) server_negotiation.cc:407] Connection header received
0809 23:08:50.981115 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:50.981117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:50.981128 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:50.981193 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:50.982155 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.982642 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.983398 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:50.983587 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:50.983643 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:50.986377 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:50.986378 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:50.986379 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:50.986387 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:50.988383 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.988775 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.988776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.988776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.988870 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:50.989177 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:50.989178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:50.989178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:50.989306 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:50.989563 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:50.989783 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:08:50.989919 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 23:08:50.991027 9989 tablet_service.cc:2908] Rejecting scan request for tablet f8e97d2bfb03406983a27119cca17db4: Uninitialized: clean time has not yet been initialized
I20260809 23:08:50.992130 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9900
I20260809 23:08:51.004099 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10142
I20260809 23:08:51.023455 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10348
I20260809 23:08:51.033912 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9736
2026-08-09T23:08:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2500 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:08:51 dist-test-slave-j2vl krb5kdc[10671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:08:51 dist-test-slave-j2vl krb5kdc[10671](info): set up 2 sockets
Aug 09 23:08:51 dist-test-slave-j2vl krb5kdc[10671](info): commencing operation
krb5kdc: starting...
W20260809 23:08:52.342062 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:08:52 dist-test-slave-j2vl krb5kdc[10671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:08:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:08:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:08:52.481627 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:34895
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:52.577700 10691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:52.577963 10691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:52.578011 10691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:52.579245 10691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:08:52.579313 10691 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 23:08:52.579335 10691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:52.579392 10691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:52.579424 10691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:08:52.579454 10691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:08:52.581005 10691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:34895
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:52.581306 10691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:52.581590 10691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:52.586743 10705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:52.586786 10702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:52.586743 10701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:52.587024 10691 server_base.cc:1034] running on GCE node
I20260809 23:08:52.587302 10691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:52.587610 10691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:52.588809 10691 hybrid_clock.cc:648] HybridClock initialized: now 1786316932588781 us; error 41 us; skew 500 ppm
Aug 09 23:08:52 dist-test-slave-j2vl krb5kdc[10671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:52.591575 10691 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:08:52.592085 10691 webserver.cc:460] Webserver started at http://127.8.141.254:39147/ using document root <none> and password file <none>
I20260809 23:08:52.592350 10691 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:52.592424 10691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:52.592602 10691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:52.593784 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6"
format_stamp: "Formatted at 2026-08-09 23:08:52 on dist-test-slave-j2vl"
server_key: "6658fc82a74deac8d775c187ebe05fab"
server_key_iv: "da6f02d15050534f7b0ae397f7216424"
server_key_version: "encryptionkey@0"
I20260809 23:08:52.594149 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6"
format_stamp: "Formatted at 2026-08-09 23:08:52 on dist-test-slave-j2vl"
server_key: "6658fc82a74deac8d775c187ebe05fab"
server_key_iv: "da6f02d15050534f7b0ae397f7216424"
server_key_version: "encryptionkey@0"
I20260809 23:08:52.595579 10691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:08:52.596438 10714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:52.596655 10691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:52.596764 10691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6"
format_stamp: "Formatted at 2026-08-09 23:08:52 on dist-test-slave-j2vl"
server_key: "6658fc82a74deac8d775c187ebe05fab"
server_key_iv: "da6f02d15050534f7b0ae397f7216424"
server_key_version: "encryptionkey@0"
I20260809 23:08:52.596853 10691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:52.607532 10691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:52.609985 10691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:52.610159 10691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:52.615612 10691 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:34895
I20260809 23:08:52.615674 10815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:34895 every 8 connection(s)
I20260809 23:08:52.616066 10691 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:08:52.616833 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:52.617671 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10691
I20260809 23:08:52.617780 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:08:52.617996 8759 external_mini_cluster.cc:1428] Setting key 4c72d6a88d67c0e2fd5febadc1ca7581
I20260809 23:08:52.620522 10818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6: Bootstrap starting.
I20260809 23:08:52.621246 10818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:52.621618 10818 log.cc:824] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:52.622645 10818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6: No bootstrap required, opened a new log
I20260809 23:08:52.625934 10818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34895 } }
I20260809 23:08:52.626101 10818 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:52.626123 10818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f61a1ee8baa469bb7f5b00f07c3fae6, State: Initialized, Role: FOLLOWER
Aug 09 23:08:52 dist-test-slave-j2vl krb5kdc[10671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:52.626287 10818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34895 } }
I20260809 23:08:52.626377 10818 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:08:52.626451 10818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:08:52.626500 10818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:52.627152 10818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34895 } }
I20260809 23:08:52.627249 10818 leader_election.cc:302] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f61a1ee8baa469bb7f5b00f07c3fae6; no voters:
I20260809 23:08:52.627494 10818 leader_election.cc:288] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:08:52.627564 10827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:52.627743 10827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [term 1 LEADER]: Becoming Leader. State: Replica: 2f61a1ee8baa469bb7f5b00f07c3fae6, State: Running, Role: LEADER
I20260809 23:08:52.627823 10818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:08:52.627837 10827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34895 } }
I20260809 23:08:52.628371 10828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34895 } } }
I20260809 23:08:52.628471 10828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:08:52.628664 10830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f61a1ee8baa469bb7f5b00f07c3fae6. Latest consensus state: current_term: 1 leader_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f61a1ee8baa469bb7f5b00f07c3fae6" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34895 } } }
I20260809 23:08:52.628742 10830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6 [sys.catalog]: This master's current role is: LEADER
I20260809 23:08:52.629460 10836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:08:52.630133 10836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:08:52.631830 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:52.618588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50604 (local address 127.8.141.254:34895)
0809 23:08:52.618779 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:52.618788 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:52.618816 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:08:52.619535 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:52.619545 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:52.619605 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:52.619743 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:52.620645 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.621766 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.622692 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.623195 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.623294 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:52.626857 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:52.626863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:52.626864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:52.626877 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:52.629524 (+ 2647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:52.630027 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:52.630031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:52.630032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:52.630140 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:52.630391 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:52.630392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:52.630392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:52.630824 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:52.631382 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:52.631464 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 23:08:52.631618 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260809 23:08:52.631845 10836 catalog_manager.cc:1347] Generated new cluster ID: 52c1385bdb8d4366bfe88fadc8a0d24e
I20260809 23:08:52.631915 10836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:08:52.650094 10836 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:08:52.650709 10836 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:08:52.655169 10836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6: Generated new TSK 0
I20260809 23:08:52.655406 10836 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:08:52.698006 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:34895
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:52.798576 10860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:52.798818 10860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:52.798849 10860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:52.800225 10860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:52.800278 10860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:52.800302 10860 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:08:52.800320 10860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:08:52.802652 10860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:34895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:52.802912 10860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:52.803120 10860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:52.808523 10867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:52.808531 10869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:52.808537 10872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:52.808917 10860 server_base.cc:1034] running on GCE node
I20260809 23:08:52.809150 10860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:52.809499 10860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:52.810729 10860 hybrid_clock.cc:648] HybridClock initialized: now 1786316932810695 us; error 53 us; skew 500 ppm
Aug 09 23:08:52 dist-test-slave-j2vl krb5kdc[10671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:52.817693 10860 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:08:52.818254 10860 webserver.cc:460] Webserver started at http://127.8.141.193:38241/ using document root <none> and password file <none>
I20260809 23:08:52.818542 10860 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:52.818619 10860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:52.818776 10860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:52.820065 10860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "33c920915d7f4d7cbf05cab2b876f6a4"
format_stamp: "Formatted at 2026-08-09 23:08:52 on dist-test-slave-j2vl"
server_key: "f8ac6eec0b838ab6e86a81c03c61cc4c"
server_key_iv: "58b67fdc658282003b331f4ac933233a"
server_key_version: "encryptionkey@0"
I20260809 23:08:52.820473 10860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "33c920915d7f4d7cbf05cab2b876f6a4"
format_stamp: "Formatted at 2026-08-09 23:08:52 on dist-test-slave-j2vl"
server_key: "f8ac6eec0b838ab6e86a81c03c61cc4c"
server_key_iv: "58b67fdc658282003b331f4ac933233a"
server_key_version: "encryptionkey@0"
I20260809 23:08:52.822266 10860 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 23:08:52.823232 10887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:52.823586 10860 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:52.823698 10860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "33c920915d7f4d7cbf05cab2b876f6a4"
format_stamp: "Formatted at 2026-08-09 23:08:52 on dist-test-slave-j2vl"
server_key: "f8ac6eec0b838ab6e86a81c03c61cc4c"
server_key_iv: "58b67fdc658282003b331f4ac933233a"
server_key_version: "encryptionkey@0"
I20260809 23:08:52.823771 10860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:52.845927 10860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:52.848444 10860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:52.848582 10860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:52.849100 10860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:52.849138 10860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:52.849164 10860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:52.849225 10860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:08:52 dist-test-slave-j2vl krb5kdc[10671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:52.857661 10860 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:33479
I20260809 23:08:52.857723 11089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:33479 every 8 connection(s)
I20260809 23:08:52.858069 10860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:08:52.860810 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:52.849928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:47147 (local address 127.8.141.254:34895)
0809 23:08:52.850006 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:52.850009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:52.850309 (+ 300us) server_negotiation.cc:407] Connection header received
0809 23:08:52.851112 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:52.851114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:52.851130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:52.851170 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:52.852299 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.852892 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.854056 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.854295 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.854376 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:52.857685 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:52.857687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:52.857687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:52.857696 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:52.859260 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:52.859698 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:52.859700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:52.859700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:52.859759 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:52.860033 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:52.860034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:52.860035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:52.860161 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:52.860235 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:52.860529 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 23:08:52.860709 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 23:08:52.861073 10901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:52.849900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34895 (local address 127.8.141.193:47147)
0809 23:08:52.850127 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:52.850183 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:52.850964 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:52.851405 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:52.851412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:52.851842 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:52.852178 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:52.852186 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.853038 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.853039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:52.853948 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:52.853951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.854400 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.854401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:52.854530 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:52.855127 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:52.855135 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:52.857572 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:52.859374 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:52.859382 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:52.859385 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:52.859619 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:52.859842 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:52.859843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:52.859843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:52.859960 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:52.860239 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:52.860247 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:52.860389 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 23:08:52.860493 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 23:08:52.860777 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":116,"threads_started":1}
I20260809 23:08:52.861694 11090 heartbeater.cc:347] Connected to a master server at 127.8.141.254:34895
I20260809 23:08:52.861793 11090 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:52.862085 11090 heartbeater.cc:511] Master 127.8.141.254:34895 requested a full tablet report, sending...
I20260809 23:08:52.862742 10739 ts_manager.cc:194] Registered new tserver with Master: 33c920915d7f4d7cbf05cab2b876f6a4 (127.8.141.193:33479)
I20260809 23:08:52.863734 10739 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:47147
I20260809 23:08:52.865988 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10860
I20260809 23:08:52.866093 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:08:52.866276 8759 external_mini_cluster.cc:1428] Setting key d28644c621a9a09cc240abea164be666
I20260809 23:08:52.869261 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:52.863005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:51963 (local address 127.8.141.254:34895)
0809 23:08:52.863062 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:52.863078 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:52.863255 (+ 177us) server_negotiation.cc:407] Connection header received
0809 23:08:52.863310 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:52.863310 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:52.863319 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:52.863364 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:52.863824 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.864353 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.865060 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.865259 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.865336 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:52.866428 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:52.866429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:52.866429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:52.866449 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:52.868054 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:52.868437 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:52.868438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:52.868439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:52.868502 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:52.868723 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:52.868724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:52.868724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:52.868837 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:52.868916 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:52.869088 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 23:08:52.869167 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 23:08:52.869279 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:52.862954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34895 (local address 127.8.141.193:51963)
0809 23:08:52.863180 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:52.863192 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:52.863255 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:52.863378 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:52.863379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:52.863584 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:52.863662 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:52.863664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.864458 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.864460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:52.864973 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:52.864975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:52.865335 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:52.865336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:52.865450 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:52.865893 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:52.865899 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:52.866322 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:52.868165 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:52.868166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:52.868166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:52.868358 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:52.868580 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:52.868580 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:52.868581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:52.868647 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:52.868907 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:52.868908 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:52.868961 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 23:08:52.869046 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 23:08:52.869165 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:08:52.898574 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:34895
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:52.991731 11108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:52.991961 11108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:52.991989 11108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:52.993237 11108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:52.993281 11108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:52.993304 11108 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:08:52.993321 11108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:08:52.994997 11108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:34895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:52.995272 11108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:52.995525 11108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:53.000949 11120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:53.000960 11115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:53.000960 11117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:53.001416 11108 server_base.cc:1034] running on GCE node
I20260809 23:08:53.001609 11108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:53.001816 11108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:53.002971 11108 hybrid_clock.cc:648] HybridClock initialized: now 1786316933002886 us; error 97 us; skew 500 ppm
Aug 09 23:08:52 dist-test-slave-j2vl krb5kdc[10671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:53.005833 11108 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:08:53.006361 11108 webserver.cc:460] Webserver started at http://127.8.141.194:39175/ using document root <none> and password file <none>
I20260809 23:08:53.006630 11108 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:53.006704 11108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:53.006857 11108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:53.008055 11108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "a499f1e9936f4e46bdcccc16d425f69b"
format_stamp: "Formatted at 2026-08-09 23:08:53 on dist-test-slave-j2vl"
server_key: "2a628288da6aacdef30dc81289981b10"
server_key_iv: "5730e17b03d4359e4fab21bf111b962e"
server_key_version: "encryptionkey@0"
I20260809 23:08:53.008555 11108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "a499f1e9936f4e46bdcccc16d425f69b"
format_stamp: "Formatted at 2026-08-09 23:08:53 on dist-test-slave-j2vl"
server_key: "2a628288da6aacdef30dc81289981b10"
server_key_iv: "5730e17b03d4359e4fab21bf111b962e"
server_key_version: "encryptionkey@0"
I20260809 23:08:53.010223 11108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:08:53.011219 11130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:53.011545 11108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:53.011684 11108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "a499f1e9936f4e46bdcccc16d425f69b"
format_stamp: "Formatted at 2026-08-09 23:08:53 on dist-test-slave-j2vl"
server_key: "2a628288da6aacdef30dc81289981b10"
server_key_iv: "5730e17b03d4359e4fab21bf111b962e"
server_key_version: "encryptionkey@0"
I20260809 23:08:53.011797 11108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:53.022903 11108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:53.025425 11108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:53.025632 11108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:53.026294 11108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:53.026351 11108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:53.026386 11108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:53.026405 11108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[10671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:53.034242 11108 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:40337
I20260809 23:08:53.034688 11108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:08:53.035240 11335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:40337 every 8 connection(s)
I20260809 23:08:53.035563 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11108
I20260809 23:08:53.035676 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:08:53.035857 8759 external_mini_cluster.cc:1428] Setting key 0048a8a2f04086f4d927e238a3b2313a
I20260809 23:08:53.037137 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.026768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:46899 (local address 127.8.141.254:34895)
0809 23:08:53.026817 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.026819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.027122 (+ 303us) server_negotiation.cc:407] Connection header received
0809 23:08:53.027746 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.027748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.027760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.027841 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.028750 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.029268 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.030460 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.030707 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.030796 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.033677 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.033679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.033679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.033689 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.035602 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.036109 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.036111 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.036111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.036174 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.036447 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.036448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.036448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.036572 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.036643 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.036935 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.037046 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:08:53.037335 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.026804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34895 (local address 127.8.141.194:46899)
0809 23:08:53.026978 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.027020 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.027630 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.027926 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.027932 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.028342 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:53.028629 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.028636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.029409 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.029410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.030359 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.030362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.030806 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.030807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.030961 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.031520 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:53.031533 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:53.033592 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:53.035704 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.035712 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.035714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.036020 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.036247 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.036248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.036249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.036370 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.036645 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:53.036653 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:53.036796 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.036910 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.037080 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260809 23:08:53.037931 11336 heartbeater.cc:347] Connected to a master server at 127.8.141.254:34895
I20260809 23:08:53.038020 11336 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:53.038225 11336 heartbeater.cc:511] Master 127.8.141.254:34895 requested a full tablet report, sending...
I20260809 23:08:53.038618 10737 ts_manager.cc:194] Registered new tserver with Master: a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194:40337)
I20260809 23:08:53.039172 10737 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:46899
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
I20260809 23:08:53.048010 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.039042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34895 (local address 127.8.141.194:49447)
0809 23:08:53.041461 (+ 2419us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.041474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.041544 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.041690 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.041691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.041928 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:53.042015 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.042018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.042733 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.042734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.043614 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.043618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.043986 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.043988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.044074 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.044541 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:53.044547 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:53.045025 (+ 478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:53.046753 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.046754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.046754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.046934 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.047179 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.047180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.047180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.047240 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.047606 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:53.047607 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:53.047693 (+ 86us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.047805 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.047925 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2386,"thread_start_us":91,"threads_started":1}
I20260809 23:08:53.048034 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.039088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49447 (local address 127.8.141.254:34895)
0809 23:08:53.039128 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.039130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.041543 (+ 2413us) server_negotiation.cc:407] Connection header received
0809 23:08:53.041605 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.041606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.041615 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.041670 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.042130 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.042625 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.043713 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.043903 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.043964 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.045129 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.045130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.045131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.045145 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.046694 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.047033 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.047034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.047035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.047093 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.047338 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.047339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.047339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.047502 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.047572 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.047874 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.047969 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:08:53.068135 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:34895
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:53.155273 11351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:53.155480 11351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:53.155504 11351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:53.156642 11351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:53.156678 11351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:53.156699 11351 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:08:53.156716 11351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:08:53.158125 11351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:34895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:53.158326 11351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:53.158519 11351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:53.163813 11367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:53.163930 11362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:53.164072 11363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:53.164247 11351 server_base.cc:1034] running on GCE node
I20260809 23:08:53.164471 11351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:53.164685 11351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:53.165850 11351 hybrid_clock.cc:648] HybridClock initialized: now 1786316933165826 us; error 40 us; skew 500 ppm
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[10671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:53.168643 11351 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:08:53.169170 11351 webserver.cc:460] Webserver started at http://127.8.141.195:42103/ using document root <none> and password file <none>
I20260809 23:08:53.169442 11351 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:53.169492 11351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:53.169585 11351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:53.170607 11351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "28062e2a154046ec8624dc0a755f0aa7"
format_stamp: "Formatted at 2026-08-09 23:08:53 on dist-test-slave-j2vl"
server_key: "e0a568f795f167812d99530b0d00d020"
server_key_iv: "2c508df1f47a7f5cc2a99da63e8e27ae"
server_key_version: "encryptionkey@0"
I20260809 23:08:53.170922 11351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "28062e2a154046ec8624dc0a755f0aa7"
format_stamp: "Formatted at 2026-08-09 23:08:53 on dist-test-slave-j2vl"
server_key: "e0a568f795f167812d99530b0d00d020"
server_key_iv: "2c508df1f47a7f5cc2a99da63e8e27ae"
server_key_version: "encryptionkey@0"
I20260809 23:08:53.172220 11351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:08:53.172914 11377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:53.173121 11351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:53.173238 11351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "28062e2a154046ec8624dc0a755f0aa7"
format_stamp: "Formatted at 2026-08-09 23:08:53 on dist-test-slave-j2vl"
server_key: "e0a568f795f167812d99530b0d00d020"
server_key_iv: "2c508df1f47a7f5cc2a99da63e8e27ae"
server_key_version: "encryptionkey@0"
I20260809 23:08:53.173326 11351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:53.182018 11351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:53.184129 11351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:53.184293 11351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:53.184902 11351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:53.184957 11351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:53.184993 11351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:53.185012 11351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[10671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316933, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:53.194568 11351 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:33223
I20260809 23:08:53.194710 11600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:33223 every 8 connection(s)
I20260809 23:08:53.195047 11351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:08:53.196007 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.185376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:48189 (local address 127.8.141.254:34895)
0809 23:08:53.185429 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.185431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.185756 (+ 325us) server_negotiation.cc:407] Connection header received
0809 23:08:53.186609 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.186610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.186623 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.186690 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.187721 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.188274 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.189324 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.189558 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.189633 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.192479 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.192481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.192481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.192491 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.194390 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.194894 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.194895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.194896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.194963 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.195236 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.195237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.195238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.195423 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.195499 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.195855 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.195942 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:08:53.196156 11403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.185389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34895 (local address 127.8.141.195:48189)
0809 23:08:53.185582 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.185637 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.186465 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.186766 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.186771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.187268 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:53.187593 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.187601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.188382 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.188383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.189216 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.189218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.189646 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.189647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.189779 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.190497 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:53.190509 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:53.192364 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:53.194485 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.194490 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.194493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.194817 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.195034 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.195035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.195035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.195169 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.195536 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:53.195543 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:53.195696 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.195792 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.195932 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260809 23:08:53.196656 11601 heartbeater.cc:347] Connected to a master server at 127.8.141.254:34895
I20260809 23:08:53.196741 11601 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:53.196938 11601 heartbeater.cc:511] Master 127.8.141.254:34895 requested a full tablet report, sending...
I20260809 23:08:53.197373 10739 ts_manager.cc:194] Registered new tserver with Master: 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223)
I20260809 23:08:53.197935 10739 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:48189
I20260809 23:08:53.203824 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.198031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:36891 (local address 127.8.141.254:34895)
0809 23:08:53.198074 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.198076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.198288 (+ 212us) server_negotiation.cc:407] Connection header received
0809 23:08:53.198348 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.198348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.198360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.198446 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.198795 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.199376 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.200324 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.200528 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.200590 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.201618 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.201620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.201620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.201627 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.202859 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.203167 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.203167 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.203168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.203220 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.203384 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.203385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.203385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.203478 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.203530 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.203701 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.203761 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:08:53.203845 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.197936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34895 (local address 127.8.141.195:36891)
0809 23:08:53.198204 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.198217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.198287 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.198449 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.198450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.198632 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:53.198720 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.198722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.199475 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.199476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.200200 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.200203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.200616 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.200618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.200689 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.201180 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:53.201192 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:53.201538 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:53.202931 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.202932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.202933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.203102 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.203282 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.203283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.203283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.203333 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.203556 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:53.203557 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:53.203601 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.203674 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.203780 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":6,"thread_start_us":184,"threads_started":1}
I20260809 23:08:53.204097 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11351
I20260809 23:08:53.204185 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:08:53.204375 8759 external_mini_cluster.cc:1428] Setting key ca8f42ddbfdb4dab07b37921272afa0a
I20260809 23:08:53.205935 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[10671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316933, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[10671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316933, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:53.235368 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.226566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50610 (local address 127.8.141.254:34895)
0809 23:08:53.226676 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.226679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.226698 (+ 19us) server_negotiation.cc:407] Connection header received
0809 23:08:53.226726 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.226727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.226738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.226797 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.227535 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.228127 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.229110 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.229313 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.229511 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.232316 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.232318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.232319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.232327 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.234045 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.234455 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.234457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.234457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.234519 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.234738 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.234739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.234740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.234872 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.234984 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.235151 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.235244 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 23:08:53.237680 10739 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:08:53.239629 10739 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:53.246942 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.242321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:40337 (local address 127.0.0.1:48294)
0809 23:08:53.242455 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.242499 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.242561 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.243111 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.243111 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.243114 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:53.243166 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.243168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.244452 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.244453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.245688 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.245691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.246581 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.246583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.246674 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.246687 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.246785 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.246844 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":86,"threads_started":1}
I20260809 23:08:53.246954 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.242761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.8.141.194:40337)
0809 23:08:53.242937 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.242943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.242954 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:08:53.242996 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.242999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.243015 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.243105 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:53.243232 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.244364 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.245795 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.246483 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.246580 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.246650 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.246803 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.246844 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260809 23:08:53.247169 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.242323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:33223 (local address 127.0.0.1:43558)
0809 23:08:53.242461 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.242504 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.242563 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.242949 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.242952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.242954 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:53.243033 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.243034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.244280 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.244281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.245777 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.245779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.246888 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.246889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.246963 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.246967 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.247013 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.247061 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":48,"thread_start_us":95,"threads_started":1}
I20260809 23:08:53.247800 11629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.242561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:33479 (local address 127.0.0.1:37282)
0809 23:08:53.242684 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.242693 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.242735 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.243515 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.243516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.243519 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:53.243586 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.243587 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.244783 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.244784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.245928 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.245931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.246954 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.246955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.247210 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.247215 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.247652 (+ 437us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.247705 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260809 23:08:53.247812 11627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.242528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43558 (local address 127.8.141.195:33223)
0809 23:08:53.242747 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.242751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.242760 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:08:53.242795 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.242797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.242838 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.242942 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:53.243145 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.244135 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.245864 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.246706 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.246787 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.246847 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.247668 (+ 821us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.247710 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":111,"threads_started":1}
I20260809 23:08:53.247782 11222 tablet_service.cc:1467] Processing CreateTablet for tablet 379d00a1bdcc4b438a34b4a7616f99d7 (DEFAULT_TABLE table=test-table [id=1012fdc8bad145f39f6fc8363c13566a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:53.248237 11222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379d00a1bdcc4b438a34b4a7616f99d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.248381 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.243066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.8.141.193:33479)
0809 23:08:53.243329 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.243333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.243342 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:08:53.243390 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.243392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.243406 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.243655 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:53.243711 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.244690 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.246076 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.246863 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.248078 (+ 1215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.248156 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.248183 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.248245 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":88,"threads_started":1}
I20260809 23:08:53.248461 11482 tablet_service.cc:1467] Processing CreateTablet for tablet 379d00a1bdcc4b438a34b4a7616f99d7 (DEFAULT_TABLE table=test-table [id=1012fdc8bad145f39f6fc8363c13566a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:53.248824 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379d00a1bdcc4b438a34b4a7616f99d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.249248 10966 tablet_service.cc:1467] Processing CreateTablet for tablet 379d00a1bdcc4b438a34b4a7616f99d7 (DEFAULT_TABLE table=test-table [id=1012fdc8bad145f39f6fc8363c13566a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:53.249594 10966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 379d00a1bdcc4b438a34b4a7616f99d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.252075 11633 tablet_bootstrap.cc:492] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Bootstrap starting.
I20260809 23:08:53.252770 11634 tablet_bootstrap.cc:492] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Bootstrap starting.
I20260809 23:08:53.252899 11633 tablet_bootstrap.cc:654] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.253299 11636 tablet_bootstrap.cc:492] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Bootstrap starting.
I20260809 23:08:53.253605 11633 log.cc:824] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:53.254087 11636 tablet_bootstrap.cc:654] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.254136 11634 tablet_bootstrap.cc:654] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.254531 11636 log.cc:824] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:53.254572 11634 log.cc:824] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:53.254788 11633 tablet_bootstrap.cc:492] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: No bootstrap required, opened a new log
I20260809 23:08:53.254869 11633 ts_tablet_manager.cc:1397] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:08:53.255441 11636 tablet_bootstrap.cc:492] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: No bootstrap required, opened a new log
I20260809 23:08:53.255498 11636 ts_tablet_manager.cc:1397] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:08:53.255744 11634 tablet_bootstrap.cc:492] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: No bootstrap required, opened a new log
I20260809 23:08:53.255852 11634 ts_tablet_manager.cc:1397] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:08:53.257071 11636 raft_consensus.cc:348] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.257211 11636 raft_consensus.cc:374] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.257117 11633 raft_consensus.cc:348] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.257234 11636 raft_consensus.cc:729] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c920915d7f4d7cbf05cab2b876f6a4, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.257292 11633 raft_consensus.cc:374] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.257376 11633 raft_consensus.cc:729] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a499f1e9936f4e46bdcccc16d425f69b, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.257442 11636 consensus_queue.cc:260] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.257525 11633 consensus_queue.cc:260] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.257803 11636 ts_tablet_manager.cc:1428] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:08:53.257941 11090 heartbeater.cc:503] Master 127.8.141.254:34895 was elected leader, sending a full tablet report...
I20260809 23:08:53.257954 11336 heartbeater.cc:503] Master 127.8.141.254:34895 was elected leader, sending a full tablet report...
I20260809 23:08:53.257885 11633 ts_tablet_manager.cc:1428] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:08:53.258539 11634 raft_consensus.cc:348] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.258769 11634 raft_consensus.cc:374] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.258821 11634 raft_consensus.cc:729] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28062e2a154046ec8624dc0a755f0aa7, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.258960 11634 consensus_queue.cc:260] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.259267 11634 ts_tablet_manager.cc:1428] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:08:53.259363 11601 heartbeater.cc:503] Master 127.8.141.254:34895 was elected leader, sending a full tablet report...
W20260809 23:08:53.285506 11338 tablet.cc:2369] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:08:53.358975 11094 tablet.cc:2369] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:08:53.371280 11644 raft_consensus.cc:482] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:53.371474 11644 raft_consensus.cc:504] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.371922 11644 leader_election.cc:288] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194:40337), 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223)
I20260809 23:08:53.375034 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.372359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:42647 (local address 127.8.141.194:40337)
0809 23:08:53.372405 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.372407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.372444 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:08:53.372530 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.372531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.372543 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.372618 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:53.372793 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.373275 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.374298 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.374657 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.374754 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.374764 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.374918 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.374958 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:08:53.375053 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.372195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:40337 (local address 127.8.141.193:42647)
0809 23:08:53.372372 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.372384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.372458 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.372624 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.372624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.372627 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:53.372702 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.372703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.373400 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.373401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.374200 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.374202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.374754 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.374755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.374840 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.374843 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.374913 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.374958 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":8,"thread_start_us":100,"threads_started":1}
I20260809 23:08:53.375125 11627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.372328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:33729 (local address 127.8.141.195:33223)
0809 23:08:53.372378 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.372380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.372415 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:08:53.372603 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.372604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.372618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.372705 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:53.372916 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.373462 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.374381 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.374758 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.374839 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.374847 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.375007 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.375048 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:08:53.375198 11648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.372177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:33223 (local address 127.8.141.193:33729)
0809 23:08:53.372346 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.372360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.372430 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.372713 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.372714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.372717 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:53.372833 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.372835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.373548 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.373549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.374271 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.374273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.374852 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.374853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.374931 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.374934 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.374999 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.375044 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260809 23:08:53.375540 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379d00a1bdcc4b438a34b4a7616f99d7" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28062e2a154046ec8624dc0a755f0aa7" is_pre_election: true
I20260809 23:08:53.375622 11259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379d00a1bdcc4b438a34b4a7616f99d7" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a499f1e9936f4e46bdcccc16d425f69b" is_pre_election: true
I20260809 23:08:53.375738 11259 raft_consensus.cc:2454] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 0.
I20260809 23:08:53.375730 11519 raft_consensus.cc:2454] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 0.
I20260809 23:08:53.376027 10888 leader_election.cc:302] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28062e2a154046ec8624dc0a755f0aa7, 33c920915d7f4d7cbf05cab2b876f6a4; no voters:
I20260809 23:08:53.376204 11644 raft_consensus.cc:2781] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:53.376281 11644 raft_consensus.cc:482] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:53.376302 11644 raft_consensus.cc:3037] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.377139 11644 raft_consensus.cc:504] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.377303 11644 leader_election.cc:288] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 election: Requested vote from peers a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194:40337), 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223)
I20260809 23:08:53.377512 11259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379d00a1bdcc4b438a34b4a7616f99d7" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a499f1e9936f4e46bdcccc16d425f69b"
I20260809 23:08:53.377565 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "379d00a1bdcc4b438a34b4a7616f99d7" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28062e2a154046ec8624dc0a755f0aa7"
I20260809 23:08:53.377617 11259 raft_consensus.cc:3037] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.377678 11519 raft_consensus.cc:3037] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.378460 11519 raft_consensus.cc:2454] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 1.
I20260809 23:08:53.378485 11259 raft_consensus.cc:2454] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 1.
I20260809 23:08:53.378684 10889 leader_election.cc:302] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33c920915d7f4d7cbf05cab2b876f6a4, a499f1e9936f4e46bdcccc16d425f69b; no voters:
I20260809 23:08:53.378844 11644 raft_consensus.cc:2781] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:53.378979 11644 raft_consensus.cc:686] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 LEADER]: Becoming Leader. State: Replica: 33c920915d7f4d7cbf05cab2b876f6a4, State: Running, Role: LEADER
I20260809 23:08:53.379074 11644 consensus_queue.cc:237] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } }
I20260809 23:08:53.380383 10736 catalog_manager.cc:5495] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33c920915d7f4d7cbf05cab2b876f6a4 (127.8.141.193). New cstate: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } health_report { overall_health: HEALTHY } } }
I20260809 23:08:53.395819 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.392930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.8.141.193:33479)
0809 23:08:53.393010 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.393011 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.393021 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:08:53.393056 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.393057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.393075 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.393141 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:53.393318 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.393998 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.394913 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.395121 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.395200 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.395438 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.395540 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.395612 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.395675 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.395697 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 23:08:53.398190 11519 raft_consensus.cc:1261] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Refusing update from remote peer 33c920915d7f4d7cbf05cab2b876f6a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:53.398190 11259 raft_consensus.cc:1261] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Refusing update from remote peer 33c920915d7f4d7cbf05cab2b876f6a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:53.398655 11644 consensus_queue.cc:1035] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:53.398854 11653 consensus_queue.cc:1035] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:53.402190 11661 mvcc.cc:205] Tried to move back new op lower bound from 7316754159194996736 to 7316754159121448960. Current Snapshot: MvccSnapshot[applied={T|T < 7316754159194996736}]
I20260809 23:08:53.402367 11665 mvcc.cc:205] Tried to move back new op lower bound from 7316754159194996736 to 7316754159121448960. Current Snapshot: MvccSnapshot[applied={T|T < 7316754159194996736}]
I20260809 23:08:53.406862 10736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:53.407044 10736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:53.408114 10736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6: Sending DeleteTablet for 3 replicas of tablet 379d00a1bdcc4b438a34b4a7616f99d7
I20260809 23:08:53.408641 11482 tablet_service.cc:1514] Processing DeleteTablet for tablet 379d00a1bdcc4b438a34b4a7616f99d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:53 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:43558
I20260809 23:08:53.408666 10966 tablet_service.cc:1514] Processing DeleteTablet for tablet 379d00a1bdcc4b438a34b4a7616f99d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:53 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:37282
I20260809 23:08:53.408648 11222 tablet_service.cc:1514] Processing DeleteTablet for tablet 379d00a1bdcc4b438a34b4a7616f99d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:53 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:48294
I20260809 23:08:53.409138 11675 tablet_replica.cc:330] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: stopping tablet replica
I20260809 23:08:53.409209 11677 tablet_replica.cc:330] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: stopping tablet replica
I20260809 23:08:53.409277 11675 raft_consensus.cc:2229] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:53.409301 11677 raft_consensus.cc:2229] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:53.409229 10736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:08:53.409411 10736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:53.409430 11675 raft_consensus.cc:2258] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:53.409430 11677 raft_consensus.cc:2258] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:53.409652 11676 tablet_replica.cc:330] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: stopping tablet replica
I20260809 23:08:53.409751 11676 raft_consensus.cc:2229] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:08:53.409884 11676 raft_consensus.cc:2258] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:53.410414 11677 ts_tablet_manager.cc:1905] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:53.410456 11676 ts_tablet_manager.cc:1905] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:53.411147 10966 tablet_service.cc:1467] Processing CreateTablet for tablet 50204577436841fa8809a965609698a3 (DEFAULT_TABLE table=test-table [id=685901ddb758454b9c91a49bf8e43045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:53.411180 11482 tablet_service.cc:1467] Processing CreateTablet for tablet 50204577436841fa8809a965609698a3 (DEFAULT_TABLE table=test-table [id=685901ddb758454b9c91a49bf8e43045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:53.411280 10966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50204577436841fa8809a965609698a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.411304 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50204577436841fa8809a965609698a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.412717 11677 ts_tablet_manager.cc:1918] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:08:53.412738 11222 tablet_service.cc:1467] Processing CreateTablet for tablet 50204577436841fa8809a965609698a3 (DEFAULT_TABLE table=test-table [id=685901ddb758454b9c91a49bf8e43045]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:53.412793 11677 log.cc:1196] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/379d00a1bdcc4b438a34b4a7616f99d7
I20260809 23:08:53.412848 11222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50204577436841fa8809a965609698a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.413095 11677 ts_tablet_manager.cc:1939] T 379d00a1bdcc4b438a34b4a7616f99d7 P a499f1e9936f4e46bdcccc16d425f69b: Deleting consensus metadata
I20260809 23:08:53.413494 11634 tablet_bootstrap.cc:492] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Bootstrap starting.
I20260809 23:08:53.414166 11634 tablet_bootstrap.cc:654] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.414484 11636 tablet_bootstrap.cc:492] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: Bootstrap starting.
I20260809 23:08:53.414386 11633 tablet_bootstrap.cc:492] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: Bootstrap starting.
I20260809 23:08:53.415032 11636 tablet_bootstrap.cc:654] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.415050 11633 tablet_bootstrap.cc:654] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.415192 11675 ts_tablet_manager.cc:1905] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:53.416103 10717 catalog_manager.cc:4841] TS a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194:40337): tablet 379d00a1bdcc4b438a34b4a7616f99d7 (table test-table [id=1012fdc8bad145f39f6fc8363c13566a]) successfully deleted
I20260809 23:08:53.416325 11633 tablet_bootstrap.cc:492] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: No bootstrap required, opened a new log
I20260809 23:08:53.416383 11633 ts_tablet_manager.cc:1397] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:08:53.416576 11633 raft_consensus.cc:348] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.416664 11633 raft_consensus.cc:374] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.416692 11633 raft_consensus.cc:729] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a499f1e9936f4e46bdcccc16d425f69b, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.416738 11633 consensus_queue.cc:260] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.416846 11633 ts_tablet_manager.cc:1428] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:53.417666 11636 tablet_bootstrap.cc:492] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: No bootstrap required, opened a new log
I20260809 23:08:53.417724 11636 ts_tablet_manager.cc:1397] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 23:08:53.417809 11634 tablet_bootstrap.cc:492] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: No bootstrap required, opened a new log
I20260809 23:08:53.417858 11634 ts_tablet_manager.cc:1397] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 23:08:53.417876 11636 raft_consensus.cc:348] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.417953 11636 raft_consensus.cc:374] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.417973 11636 raft_consensus.cc:729] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c920915d7f4d7cbf05cab2b876f6a4, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.418020 11636 consensus_queue.cc:260] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.418030 11634 raft_consensus.cc:348] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.418092 11675 ts_tablet_manager.cc:1918] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:08:53.418114 11636 ts_tablet_manager.cc:1428] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:53.418103 11634 raft_consensus.cc:374] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.418155 11634 raft_consensus.cc:729] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28062e2a154046ec8624dc0a755f0aa7, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.418180 11675 log.cc:1196] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/379d00a1bdcc4b438a34b4a7616f99d7
I20260809 23:08:53.418205 11634 consensus_queue.cc:260] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.418318 11634 ts_tablet_manager.cc:1428] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:08:53.418479 11675 ts_tablet_manager.cc:1939] T 379d00a1bdcc4b438a34b4a7616f99d7 P 28062e2a154046ec8624dc0a755f0aa7: Deleting consensus metadata
I20260809 23:08:53.419065 10716 catalog_manager.cc:4841] TS 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223): tablet 379d00a1bdcc4b438a34b4a7616f99d7 (table test-table [id=1012fdc8bad145f39f6fc8363c13566a]) successfully deleted
I20260809 23:08:53.420102 11676 ts_tablet_manager.cc:1918] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:08:53.420186 11676 log.cc:1196] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/379d00a1bdcc4b438a34b4a7616f99d7
I20260809 23:08:53.420459 11676 ts_tablet_manager.cc:1939] T 379d00a1bdcc4b438a34b4a7616f99d7 P 33c920915d7f4d7cbf05cab2b876f6a4: Deleting consensus metadata
I20260809 23:08:53.421177 10717 catalog_manager.cc:4841] TS 33c920915d7f4d7cbf05cab2b876f6a4 (127.8.141.193:33479): tablet 379d00a1bdcc4b438a34b4a7616f99d7 (table test-table [id=1012fdc8bad145f39f6fc8363c13566a]) successfully deleted
W20260809 23:08:53.445729 11603 tablet.cc:2369] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:08:53.451531 11646 raft_consensus.cc:482] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:53.451659 11646 raft_consensus.cc:504] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.452067 11646 leader_election.cc:288] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33c920915d7f4d7cbf05cab2b876f6a4 (127.8.141.193:33479), 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223)
I20260809 23:08:53.455662 11627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.452380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:35753 (local address 127.8.141.195:33223)
0809 23:08:53.452428 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.452430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.452504 (+ 74us) server_negotiation.cc:407] Connection header received
0809 23:08:53.452568 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.452569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.452579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.452660 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:53.452942 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.453518 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.454751 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.455154 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.455249 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.455262 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.455524 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.455580 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:08:53.455703 11682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.452241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:33223 (local address 127.8.141.194:35753)
0809 23:08:53.452439 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.452447 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.452499 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.452814 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.452815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.452816 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:53.452865 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.452866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.453642 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.453643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.454608 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.454613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.455271 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.455273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.455419 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.455425 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.455499 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.455547 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":21,"thread_start_us":67,"threads_started":1}
I20260809 23:08:53.456104 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50204577436841fa8809a965609698a3" candidate_uuid: "a499f1e9936f4e46bdcccc16d425f69b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28062e2a154046ec8624dc0a755f0aa7" is_pre_election: true
I20260809 23:08:53.456218 11519 raft_consensus.cc:2454] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a499f1e9936f4e46bdcccc16d425f69b in term 0.
I20260809 23:08:53.456415 11131 leader_election.cc:302] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28062e2a154046ec8624dc0a755f0aa7, a499f1e9936f4e46bdcccc16d425f69b; no voters:
I20260809 23:08:53.456569 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.452472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:41527 (local address 127.8.141.193:33479)
0809 23:08:53.452515 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.452517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.452525 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:08:53.452554 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.452555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.452565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.452643 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:53.452824 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.453598 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.455431 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.456025 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.456091 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.456100 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.456454 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.456503 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:08:53.456629 11646 raft_consensus.cc:2781] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:53.456698 11646 raft_consensus.cc:482] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:53.456728 11646 raft_consensus.cc:3037] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.457971 11646 raft_consensus.cc:504] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.458153 11646 leader_election.cc:288] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [CANDIDATE]: Term 1 election: Requested vote from peers 33c920915d7f4d7cbf05cab2b876f6a4 (127.8.141.193:33479), 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223)
I20260809 23:08:53.458299 11146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.452241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:33479 (local address 127.8.141.194:41527)
0809 23:08:53.452298 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.452311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.452355 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.452660 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.452661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.452663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:53.452749 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.452751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.453709 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.453710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.455293 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.455297 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.456136 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.456137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.456257 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.456262 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.456315 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.456364 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:08:53.458518 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50204577436841fa8809a965609698a3" candidate_uuid: "a499f1e9936f4e46bdcccc16d425f69b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28062e2a154046ec8624dc0a755f0aa7"
I20260809 23:08:53.458621 11519 raft_consensus.cc:3037] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.458662 11006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50204577436841fa8809a965609698a3" candidate_uuid: "a499f1e9936f4e46bdcccc16d425f69b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c920915d7f4d7cbf05cab2b876f6a4"
I20260809 23:08:53.458784 11006 raft_consensus.cc:3037] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.458848 11007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50204577436841fa8809a965609698a3" candidate_uuid: "a499f1e9936f4e46bdcccc16d425f69b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" is_pre_election: true
I20260809 23:08:53.459594 11519 raft_consensus.cc:2454] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a499f1e9936f4e46bdcccc16d425f69b in term 1.
I20260809 23:08:53.459667 11006 raft_consensus.cc:2454] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a499f1e9936f4e46bdcccc16d425f69b in term 1.
I20260809 23:08:53.459790 11131 leader_election.cc:302] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28062e2a154046ec8624dc0a755f0aa7, a499f1e9936f4e46bdcccc16d425f69b; no voters:
I20260809 23:08:53.459895 11646 raft_consensus.cc:2781] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:53.460054 11646 raft_consensus.cc:686] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 1 LEADER]: Becoming Leader. State: Replica: a499f1e9936f4e46bdcccc16d425f69b, State: Running, Role: LEADER
I20260809 23:08:53.460124 11646 consensus_queue.cc:237] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.461241 10737 catalog_manager.cc:5495] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b reported cstate change: term changed from 0 to 1, leader changed from <none> to a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194). New cstate: current_term: 1 leader_uuid: "a499f1e9936f4e46bdcccc16d425f69b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } health_report { overall_health: UNKNOWN } } }
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[10671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316932, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:53.503391 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.494572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50614 (local address 127.8.141.254:34895)
0809 23:08:53.494643 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.494645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.494656 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:08:53.494771 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.494772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.494782 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.494861 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.495477 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.495980 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.496749 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.496960 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.497039 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.499770 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.499771 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.499771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.499783 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.501587 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.502058 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.502059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.502060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.502122 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.502728 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.502730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.502730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.502874 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.502952 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.503153 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.503272 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 23:08:53.503477 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.494263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34895 (local address 127.0.0.1:50614)
0809 23:08:53.494564 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.494576 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.494670 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.494859 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.494860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.495203 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:53.495344 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.495349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.496068 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.496070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.496657 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.496659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.497036 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.497037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.497099 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.498009 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:53.498015 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:53.499654 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:53.501720 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.501730 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.501733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.501964 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.502387 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:53.502389 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:53.502389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:53.502609 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:53.502984 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:53.502995 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:53.503045 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.503117 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.503341 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":95,"thread_start_us":135,"threads_started":1}
I20260809 23:08:53.507449 10737 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:50614:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 23:08:53.507833 10737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:53.510051 11482 tablet_service.cc:1467] Processing CreateTablet for tablet 921d1ed262a64ceeb56cda466fd8e93d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=48e6432f500140619613df597481280c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:08:53.510190 11222 tablet_service.cc:1467] Processing CreateTablet for tablet 921d1ed262a64ceeb56cda466fd8e93d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=48e6432f500140619613df597481280c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:08:53.510182 10966 tablet_service.cc:1467] Processing CreateTablet for tablet 921d1ed262a64ceeb56cda466fd8e93d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=48e6432f500140619613df597481280c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:08:53.510329 11222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 921d1ed262a64ceeb56cda466fd8e93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.510334 11482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 921d1ed262a64ceeb56cda466fd8e93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.510357 10966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 921d1ed262a64ceeb56cda466fd8e93d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:53.512117 11633 tablet_bootstrap.cc:492] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: Bootstrap starting.
I20260809 23:08:53.512166 11636 tablet_bootstrap.cc:492] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: Bootstrap starting.
I20260809 23:08:53.512552 11633 tablet_bootstrap.cc:654] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.512602 11636 tablet_bootstrap.cc:654] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.513048 11634 tablet_bootstrap.cc:492] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: Bootstrap starting.
I20260809 23:08:53.513628 11636 tablet_bootstrap.cc:492] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: No bootstrap required, opened a new log
I20260809 23:08:53.513688 11636 ts_tablet_manager.cc:1397] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:08:53.513824 11634 tablet_bootstrap.cc:654] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:53.513863 11636 raft_consensus.cc:348] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.513947 11636 raft_consensus.cc:374] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.513970 11636 raft_consensus.cc:729] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33c920915d7f4d7cbf05cab2b876f6a4, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.514019 11636 consensus_queue.cc:260] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.514124 11636 ts_tablet_manager.cc:1428] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:08:53.514151 11669 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.514235 11669 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:53.514938 11633 tablet_bootstrap.cc:492] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: No bootstrap required, opened a new log
I20260809 23:08:53.514997 11633 ts_tablet_manager.cc:1397] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 23:08:53.515157 11633 raft_consensus.cc:348] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.515242 11633 raft_consensus.cc:374] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.515265 11633 raft_consensus.cc:729] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a499f1e9936f4e46bdcccc16d425f69b, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.515319 11633 consensus_queue.cc:260] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.515421 11633 ts_tablet_manager.cc:1428] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:53.515610 11684 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.515702 11684 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:53.515962 11634 tablet_bootstrap.cc:492] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: No bootstrap required, opened a new log
I20260809 23:08:53.516021 11634 ts_tablet_manager.cc:1397] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 23:08:53.516198 11634 raft_consensus.cc:348] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.516300 11634 raft_consensus.cc:374] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:53.516330 11634 raft_consensus.cc:729] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28062e2a154046ec8624dc0a755f0aa7, State: Initialized, Role: FOLLOWER
I20260809 23:08:53.516379 11634 consensus_queue.cc:260] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.516497 11634 ts_tablet_manager.cc:1428] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:53.516748 11647 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.516878 11647 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:53.631140 11669 raft_consensus.cc:482] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:53.631269 11669 raft_consensus.cc:504] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.631461 11669 leader_election.cc:288] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194:40337), 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223)
I20260809 23:08:53.631743 11259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "921d1ed262a64ceeb56cda466fd8e93d" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a499f1e9936f4e46bdcccc16d425f69b" is_pre_election: true
I20260809 23:08:53.631763 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "921d1ed262a64ceeb56cda466fd8e93d" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28062e2a154046ec8624dc0a755f0aa7" is_pre_election: true
I20260809 23:08:53.631856 11259 raft_consensus.cc:2454] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 0.
I20260809 23:08:53.631907 11519 raft_consensus.cc:2454] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 0.
I20260809 23:08:53.632108 10888 leader_election.cc:302] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28062e2a154046ec8624dc0a755f0aa7, 33c920915d7f4d7cbf05cab2b876f6a4; no voters:
I20260809 23:08:53.632226 11669 raft_consensus.cc:2781] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:53.632274 11669 raft_consensus.cc:482] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:53.632296 11669 raft_consensus.cc:3037] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.633059 11669 raft_consensus.cc:504] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.633214 11669 leader_election.cc:288] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 election: Requested vote from peers a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194:40337), 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223)
I20260809 23:08:53.633486 11519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "921d1ed262a64ceeb56cda466fd8e93d" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28062e2a154046ec8624dc0a755f0aa7"
I20260809 23:08:53.633495 11259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "921d1ed262a64ceeb56cda466fd8e93d" candidate_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a499f1e9936f4e46bdcccc16d425f69b"
I20260809 23:08:53.633582 11259 raft_consensus.cc:3037] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.633582 11519 raft_consensus.cc:3037] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:53.634438 11259 raft_consensus.cc:2454] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 1.
I20260809 23:08:53.634438 11519 raft_consensus.cc:2454] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33c920915d7f4d7cbf05cab2b876f6a4 in term 1.
I20260809 23:08:53.634647 10888 leader_election.cc:302] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28062e2a154046ec8624dc0a755f0aa7, 33c920915d7f4d7cbf05cab2b876f6a4; no voters:
I20260809 23:08:53.634770 11669 raft_consensus.cc:2781] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:53.634860 11669 raft_consensus.cc:686] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 LEADER]: Becoming Leader. State: Replica: 33c920915d7f4d7cbf05cab2b876f6a4, State: Running, Role: LEADER
I20260809 23:08:53.634948 11669 consensus_queue.cc:237] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } }
I20260809 23:08:53.635329 11644 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: TxnStatusTablet state changed. Reason: New leader 33c920915d7f4d7cbf05cab2b876f6a4. Latest consensus state: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.635430 11644 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:08:53.635629 11702 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 23:08:53.635792 11702 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 23:08:53.635795 10737 catalog_manager.cc:5495] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33c920915d7f4d7cbf05cab2b876f6a4 (127.8.141.193). New cstate: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } health_report { overall_health: UNKNOWN } } }
I20260809 23:08:53.669552 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.667084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:33479 (local address 127.0.0.1:37298)
0809 23:08:53.667167 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.667180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.667246 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.667403 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.667404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.667405 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:08:53.667459 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.667461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.668107 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.668109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.668679 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.668681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.669051 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.669052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.669120 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.669143 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.669371 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.669382 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.669430 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.669483 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 23:08:53.669546 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.667162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.8.141.193:33479)
0809 23:08:53.667220 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.667223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.667234 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:08:53.667327 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.667328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.667340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.667393 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:53.667519 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.668005 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.668781 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.668975 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.669050 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.669237 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.669279 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.669341 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.669439 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.669462 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:08:53.670913 11259 raft_consensus.cc:1261] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Refusing update from remote peer 33c920915d7f4d7cbf05cab2b876f6a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:53.670909 11519 raft_consensus.cc:1261] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Refusing update from remote peer 33c920915d7f4d7cbf05cab2b876f6a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:53.671170 11669 consensus_queue.cc:1035] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:53.671283 11644 consensus_queue.cc:1035] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:53.671887 11684 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: TxnStatusTablet state changed. Reason: New leader 33c920915d7f4d7cbf05cab2b876f6a4. Latest consensus state: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.672005 11684 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:53.672317 11647 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: TxnStatusTablet state changed. Reason: New leader 33c920915d7f4d7cbf05cab2b876f6a4. Latest consensus state: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.672407 11647 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:53.672468 11669 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.672560 11669 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:08:53.673312 11684 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.673425 11684 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P a499f1e9936f4e46bdcccc16d425f69b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:53.674491 11644 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.674583 11644 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P 33c920915d7f4d7cbf05cab2b876f6a4: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:08:53.675375 11662 mvcc.cc:205] Tried to move back new op lower bound from 7316754160313798656 to 7316754160169099264. Current Snapshot: MvccSnapshot[applied={T|T < 7316754160313798656}]
I20260809 23:08:53.675552 11647 tablet_replica.cc:439] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 } } peers { permanent_uuid: "a499f1e9936f4e46bdcccc16d425f69b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 40337 } } peers { permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 } } }
I20260809 23:08:53.675638 11647 tablet_replica.cc:442] T 921d1ed262a64ceeb56cda466fd8e93d P 28062e2a154046ec8624dc0a755f0aa7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:53.677883 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.675019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.8.141.194:40337)
0809 23:08:53.675056 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.675058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.675066 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:08:53.675103 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.675104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.675124 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.675196 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:53.675353 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.676048 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.677004 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.677229 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.677308 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.677514 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.677636 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.677715 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.677771 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.677792 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:08:53.683286 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.680657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:57093 (local address 127.8.141.193:33479)
0809 23:08:53.680701 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.680702 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.680993 (+ 291us) server_negotiation.cc:407] Connection header received
0809 23:08:53.681073 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.681074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.681086 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.681130 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:53.681280 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.681768 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.682560 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.682757 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.682850 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.683030 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.683067 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.683128 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.683189 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.683211 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:08:53.683466 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.680549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:33479 (local address 127.8.141.194:57093)
0809 23:08:53.680908 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.680920 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.681008 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.681141 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.681142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.681143 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:08:53.681215 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.681217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.681879 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.681880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.682473 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.682475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.682849 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.682850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.682922 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.682935 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.683129 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.683132 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.683184 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.683227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":227,"threads_started":1}
I20260809 23:08:53.685839 11519 raft_consensus.cc:1261] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Refusing update from remote peer a499f1e9936f4e46bdcccc16d425f69b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:53.685910 11006 raft_consensus.cc:1261] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Refusing update from remote peer a499f1e9936f4e46bdcccc16d425f69b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:53.686087 11646 consensus_queue.cc:1035] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [LEADER]: Connected to new peer: Peer: permanent_uuid: "28062e2a154046ec8624dc0a755f0aa7" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 33223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:53.686225 11684 consensus_queue.cc:1035] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [LEADER]: Connected to new peer: Peer: permanent_uuid: "33c920915d7f4d7cbf05cab2b876f6a4" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 33479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:53.696285 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.692792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:53417 (local address 127.8.141.194:40337)
0809 23:08:53.692831 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.692834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.693042 (+ 208us) server_negotiation.cc:407] Connection header received
0809 23:08:53.693068 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.693069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.693079 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.693144 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:53.693227 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.693974 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.695522 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.695720 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.695785 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.695986 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.696026 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.696122 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.696180 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.696204 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:08:53.696303 11720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.692753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:40337 (local address 127.8.141.193:53417)
0809 23:08:53.692968 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:53.692978 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:53.693032 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:53.693147 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:53.693147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:53.693148 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:08:53.693191 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.693192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.694068 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.694069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.695415 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:53.695418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.695804 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.695805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:53.695861 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.695879 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:08:53.696125 (+ 246us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:08:53.696128 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:53.696178 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 23:08:53.696226 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
I20260809 23:08:53.706314 10737 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:53.706475 10737 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:53.707226 10737 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f61a1ee8baa469bb7f5b00f07c3fae6: Sending DeleteTablet for 3 replicas of tablet 50204577436841fa8809a965609698a3
I20260809 23:08:53.707604 11482 tablet_service.cc:1514] Processing DeleteTablet for tablet 50204577436841fa8809a965609698a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:53 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:43558
I20260809 23:08:53.707659 11222 tablet_service.cc:1514] Processing DeleteTablet for tablet 50204577436841fa8809a965609698a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:53 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:48294
I20260809 23:08:53.707713 11675 tablet_replica.cc:330] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: stopping tablet replica
I20260809 23:08:53.707764 11675 raft_consensus.cc:2229] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:53.707743 11677 tablet_replica.cc:330] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: stopping tablet replica
I20260809 23:08:53.707803 11675 raft_consensus.cc:2258] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:53.707814 11677 raft_consensus.cc:2229] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:08:53.707890 11677 raft_consensus.cc:2258] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:53.708078 10966 tablet_service.cc:1514] Processing DeleteTablet for tablet 50204577436841fa8809a965609698a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:53 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:37282
I20260809 23:08:53.708199 11676 tablet_replica.cc:330] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: stopping tablet replica
I20260809 23:08:53.708248 11676 raft_consensus.cc:2229] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:53.708315 11676 raft_consensus.cc:2258] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:53.708452 11675 ts_tablet_manager.cc:1905] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:53.710002 11676 ts_tablet_manager.cc:1905] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:53.710060 11677 ts_tablet_manager.cc:1905] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:53.711830 11675 ts_tablet_manager.cc:1918] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:53.711934 11675 log.cc:1196] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/50204577436841fa8809a965609698a3
I20260809 23:08:53.712162 11675 ts_tablet_manager.cc:1939] T 50204577436841fa8809a965609698a3 P 28062e2a154046ec8624dc0a755f0aa7: Deleting consensus metadata
I20260809 23:08:53.712541 11677 ts_tablet_manager.cc:1918] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:53.712565 10716 catalog_manager.cc:4841] TS 28062e2a154046ec8624dc0a755f0aa7 (127.8.141.195:33223): tablet 50204577436841fa8809a965609698a3 (table test-table [id=685901ddb758454b9c91a49bf8e43045]) successfully deleted
I20260809 23:08:53.712604 11677 log.cc:1196] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/50204577436841fa8809a965609698a3
I20260809 23:08:53.712832 11677 ts_tablet_manager.cc:1939] T 50204577436841fa8809a965609698a3 P a499f1e9936f4e46bdcccc16d425f69b: Deleting consensus metadata
I20260809 23:08:53.713282 11676 ts_tablet_manager.cc:1918] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:53.713351 11676 log.cc:1196] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/50204577436841fa8809a965609698a3
I20260809 23:08:53.713325 10717 catalog_manager.cc:4841] TS a499f1e9936f4e46bdcccc16d425f69b (127.8.141.194:40337): tablet 50204577436841fa8809a965609698a3 (table test-table [id=685901ddb758454b9c91a49bf8e43045]) successfully deleted
I20260809 23:08:53.713599 11676 ts_tablet_manager.cc:1939] T 50204577436841fa8809a965609698a3 P 33c920915d7f4d7cbf05cab2b876f6a4: Deleting consensus metadata
I20260809 23:08:53.713965 10717 catalog_manager.cc:4841] TS 33c920915d7f4d7cbf05cab2b876f6a4 (127.8.141.193:33479): tablet 50204577436841fa8809a965609698a3 (table test-table [id=685901ddb758454b9c91a49bf8e43045]) successfully deleted
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[10671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316933, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:08:53.728820 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.717735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.8.141.193:33479)
0809 23:08:53.717789 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.717791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.718070 (+ 279us) server_negotiation.cc:407] Connection header received
0809 23:08:53.718186 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.718187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.718197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.718263 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.719208 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.719959 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.721619 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.721848 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.721943 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.725172 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.725184 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.725185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.725231 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.727437 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.727884 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.727888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.727889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.727958 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.728165 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.728166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.728167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.728366 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.728514 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.728612 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.728706 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 23:08:53.729240 11075 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37308
I20260809 23:08:53.737046 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:53.730325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.8.141.254:34895)
0809 23:08:53.730373 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:53.730376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:53.730657 (+ 281us) server_negotiation.cc:407] Connection header received
0809 23:08:53.730733 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:53.730734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:53.730746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:53.730802 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:53.731350 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.731855 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.732543 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:53.732755 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:53.732832 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:53.734325 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:53.734327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:53.734327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:53.734335 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:53.735830 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.736201 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.736202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.736203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.736267 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:53.736490 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:53.736491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:53.736492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:53.736619 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:53.736732 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:53.736835 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 23:08:53.736924 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 23:08:53.737597 10737 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50620
I20260809 23:08:53.738361 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10860
I20260809 23:08:53.780815 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11108
I20260809 23:08:53.799933 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11351
I20260809 23:08:53.825987 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10691
2026-08-09T23:08:53Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2792 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[11736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[11736](info): set up 2 sockets
Aug 09 23:08:53 dist-test-slave-j2vl krb5kdc[11736](info): commencing operation
krb5kdc: starting...
W20260809 23:08:55.165757 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:08:55 dist-test-slave-j2vl krb5kdc[11736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:08:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:08:55.302551 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33861
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:33861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:55.403565 11761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:55.403867 11761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:55.403911 11761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:55.405215 11761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:08:55.405294 11761 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 23:08:55.405321 11761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:55.405375 11761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:55.405408 11761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:08:55.405437 11761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:08:55.407094 11761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:33861
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:55.407409 11761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:55.407667 11761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:55.413393 11780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:55.413520 11771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:55.413862 11773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:55.413892 11761 server_base.cc:1034] running on GCE node
I20260809 23:08:55.414270 11761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:55.414615 11761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:55.415809 11761 hybrid_clock.cc:648] HybridClock initialized: now 1786316935415790 us; error 41 us; skew 500 ppm
Aug 09 23:08:55 dist-test-slave-j2vl krb5kdc[11736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:55.419021 11761 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:08:55.419520 11761 webserver.cc:460] Webserver started at http://127.8.141.254:33239/ using document root <none> and password file <none>
I20260809 23:08:55.419763 11761 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:55.419835 11761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:55.419996 11761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:55.421134 11761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "67e9e021bec94aedb1f925c2e8c33634"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "084ed7eb1f33cd99a41d23f4f57acd76"
server_key_iv: "8553e4222d777aa47ae61f8497617fa3"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.421562 11761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "67e9e021bec94aedb1f925c2e8c33634"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "084ed7eb1f33cd99a41d23f4f57acd76"
server_key_iv: "8553e4222d777aa47ae61f8497617fa3"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.423095 11761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:55.424065 11793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:55.424278 11761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:55.424417 11761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "67e9e021bec94aedb1f925c2e8c33634"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "084ed7eb1f33cd99a41d23f4f57acd76"
server_key_iv: "8553e4222d777aa47ae61f8497617fa3"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.424499 11761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:55.436509 11761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:55.439067 11761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:55.439283 11761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:55.444738 11761 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:33861
I20260809 23:08:55.444876 11904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:33861 every 8 connection(s)
I20260809 23:08:55.445503 11761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:08:55.446621 11907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:55.449113 11907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Bootstrap starting.
I20260809 23:08:55.449719 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11761
I20260809 23:08:55.449829 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:08:55.449894 11907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:55.450053 8759 external_mini_cluster.cc:1428] Setting key 2264fdc13519e7b38e3709dedf50e75c
I20260809 23:08:55.450229 11907 log.cc:824] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:55.451043 11907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: No bootstrap required, opened a new log
I20260809 23:08:55.453287 11907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e9e021bec94aedb1f925c2e8c33634" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33861 } }
I20260809 23:08:55.453540 11907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:55.453594 11907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67e9e021bec94aedb1f925c2e8c33634, State: Initialized, Role: FOLLOWER
I20260809 23:08:55.453727 11907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e9e021bec94aedb1f925c2e8c33634" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33861 } }
I20260809 23:08:55.453830 11907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:08:55.453876 11907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:08:55.453940 11907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:55.454629 11907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e9e021bec94aedb1f925c2e8c33634" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33861 } }
I20260809 23:08:55.454773 11907 leader_election.cc:302] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67e9e021bec94aedb1f925c2e8c33634; no voters:
I20260809 23:08:55.454991 11907 leader_election.cc:288] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:08:55.455044 11916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:55.455204 11916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [term 1 LEADER]: Becoming Leader. State: Replica: 67e9e021bec94aedb1f925c2e8c33634, State: Running, Role: LEADER
I20260809 23:08:55.455299 11907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:08:55.455302 11916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e9e021bec94aedb1f925c2e8c33634" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33861 } }
I20260809 23:08:55.455703 11918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67e9e021bec94aedb1f925c2e8c33634. Latest consensus state: current_term: 1 leader_uuid: "67e9e021bec94aedb1f925c2e8c33634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e9e021bec94aedb1f925c2e8c33634" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33861 } } }
I20260809 23:08:55.455786 11918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [sys.catalog]: This master's current role is: LEADER
I20260809 23:08:55.455978 11919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67e9e021bec94aedb1f925c2e8c33634" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67e9e021bec94aedb1f925c2e8c33634" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33861 } } }
I20260809 23:08:55.456063 11928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:08:55.456655 11928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:08:55.457451 11919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634 [sys.catalog]: This master's current role is: LEADER
Aug 09 23:08:55 dist-test-slave-j2vl krb5kdc[11736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:55.458986 11928 catalog_manager.cc:1347] Generated new cluster ID: 793de234af0c446988b2404afaf5b933
I20260809 23:08:55.459041 11928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:08:55.473109 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.450616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.8.141.254:33861)
0809 23:08:55.450804 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:55.450815 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:55.450850 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:08:55.451417 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:55.451425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:55.451479 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:55.451636 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:55.452471 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.453476 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.454412 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.454947 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.455079 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.458578 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:55.458582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:55.458582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:55.458596 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:55.471147 (+ 12551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.471628 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.471633 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.471633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.471734 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.471962 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.471963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.471963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.472233 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:55.472402 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:55.472686 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 23:08:55.472867 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260809 23:08:55.475601 11928 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:08:55.476590 11928 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:08:55.482372 11928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Generated new TSK 0
I20260809 23:08:55.482620 11928 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:08:55.526505 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:33861
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:55.619081 11956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:55.619325 11956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:55.619354 11956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:55.620752 11956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:55.620819 11956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:55.620846 11956 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 23:08:55.620864 11956 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:08:55.620980 11956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:08:55.623013 11956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:33861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:55.623328 11956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:55.623607 11956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:55.629469 11968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:55.629915 11972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:55.629875 11969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:55.629946 11956 server_base.cc:1034] running on GCE node
I20260809 23:08:55.630317 11956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:55.630613 11956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:55.631846 11956 hybrid_clock.cc:648] HybridClock initialized: now 1786316935631793 us; error 60 us; skew 500 ppm
Aug 09 23:08:55 dist-test-slave-j2vl krb5kdc[11736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:55.635519 11956 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:08:55.636053 11956 webserver.cc:460] Webserver started at http://127.8.141.193:35207/ using document root <none> and password file <none>
I20260809 23:08:55.636432 11956 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:55.636519 11956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:55.636677 11956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:55.637948 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "d8240c635f2141d4a86d1ee63c0528b1"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "72814c54a5fff4c5092b8247c1e1e953"
server_key_iv: "166cdafa06d5aed32ef9d3becfa60164"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.638324 11956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "d8240c635f2141d4a86d1ee63c0528b1"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "72814c54a5fff4c5092b8247c1e1e953"
server_key_iv: "166cdafa06d5aed32ef9d3becfa60164"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.639895 11956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:08:55.641199 11988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:55.642060 11956 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 23:08:55.642207 11956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "d8240c635f2141d4a86d1ee63c0528b1"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "72814c54a5fff4c5092b8247c1e1e953"
server_key_iv: "166cdafa06d5aed32ef9d3becfa60164"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.642302 11956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:55.654176 11956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:55.656916 11956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:55.657107 11956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:55.657850 11956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:55.657933 11956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:55.657975 11956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:55.658022 11956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:08:55 dist-test-slave-j2vl krb5kdc[11736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:55.665874 11956 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:37167
I20260809 23:08:55.665944 12193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:37167 every 8 connection(s)
I20260809 23:08:55.666349 11956 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:08:55.668915 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.658265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:40599 (local address 127.8.141.254:33861)
0809 23:08:55.658327 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:55.658329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:55.658582 (+ 253us) server_negotiation.cc:407] Connection header received
0809 23:08:55.659214 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:55.659215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:55.659229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:55.659292 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:55.660196 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.661062 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.662331 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.662703 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.662804 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.665732 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:55.665734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:55.665734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:55.665743 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:55.667354 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.667806 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.667808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.667808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.667868 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.668208 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.668209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.668209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.668350 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:55.668418 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:55.668725 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 23:08:55.668843 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:08:55.669004 12008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.658259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33861 (local address 127.8.141.193:40599)
0809 23:08:55.658441 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:55.658483 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:55.659097 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:55.659369 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:55.659374 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:55.659756 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:55.660076 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.660083 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.661213 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.661214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.662207 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.662210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.662830 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.662832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.663014 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.663680 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:55.663695 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:55.665599 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:55.667445 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.667453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.667454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.667705 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.667997 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.667997 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.667998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.668119 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.668426 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:55.668432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:55.668572 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 23:08:55.668664 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:08:55.668803 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260809 23:08:55.669597 12196 heartbeater.cc:347] Connected to a master server at 127.8.141.254:33861
I20260809 23:08:55.669716 12196 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:55.669911 12196 heartbeater.cc:511] Master 127.8.141.254:33861 requested a full tablet report, sending...
I20260809 23:08:55.670467 11818 ts_manager.cc:194] Registered new tserver with Master: d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167)
I20260809 23:08:55.671326 11818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:40599
I20260809 23:08:55.673996 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11956
I20260809 23:08:55.674106 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:08:55.674312 8759 external_mini_cluster.cc:1428] Setting key 58ab667e8fd5deef2301a86debcbc379
I20260809 23:08:55.677512 12207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.670702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33861 (local address 127.8.141.193:50837)
0809 23:08:55.670830 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:55.670841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:55.670902 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:55.671028 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:55.671029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:55.671190 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:55.671244 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.671244 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.672053 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.672055 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.672831 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.672834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.673225 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.673226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.673286 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.673814 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:55.673822 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:55.674246 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:55.676131 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.676133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.676133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.676404 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.676694 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.676695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.676696 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.676773 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.677167 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:55.677167 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:55.677215 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 23:08:55.677272 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 23:08:55.677426 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260809 23:08:55.677595 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.670774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:50837 (local address 127.8.141.254:33861)
0809 23:08:55.670811 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:55.670813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:55.670902 (+ 89us) server_negotiation.cc:407] Connection header received
0809 23:08:55.670958 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:55.670958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:55.670969 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:55.671024 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:55.671304 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.671941 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.672933 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.673144 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.673218 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.674343 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:55.674344 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:55.674344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:55.674380 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:55.676020 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.676536 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.676537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.676537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.676603 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.676896 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.676898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.676898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.677088 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:55.677173 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:55.677328 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 23:08:55.677457 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:08:55.712550 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:33861
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:55.803599 12210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:55.803879 12210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:55.803920 12210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:55.805109 12210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:55.805178 12210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:55.805214 12210 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 23:08:55.805243 12210 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:08:55.805275 12210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:08:55.806808 12210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:33861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:55.807098 12210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:55.807354 12210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:55.813324 12222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:55.813390 12210 server_base.cc:1034] running on GCE node
W20260809 23:08:55.813306 12219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:55.813313 12218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:55.813879 12210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:55.814119 12210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:55.815308 12210 hybrid_clock.cc:648] HybridClock initialized: now 1786316935815297 us; error 35 us; skew 500 ppm
Aug 09 23:08:55 dist-test-slave-j2vl krb5kdc[11736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:55.818193 12210 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:08:55.818696 12210 webserver.cc:460] Webserver started at http://127.8.141.194:38931/ using document root <none> and password file <none>
I20260809 23:08:55.818950 12210 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:55.819001 12210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:55.819141 12210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:55.820303 12210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "f590b7a531b14c7aa2e4db2c7a1715be"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "85f0abf670e10928c18e38b952a49a89"
server_key_iv: "fc28affb1122df3d7388d5a70cee0ae5"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.820694 12210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "f590b7a531b14c7aa2e4db2c7a1715be"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "85f0abf670e10928c18e38b952a49a89"
server_key_iv: "fc28affb1122df3d7388d5a70cee0ae5"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.822340 12210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:08:55.823236 12245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:55.823603 12210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:55.823697 12210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "f590b7a531b14c7aa2e4db2c7a1715be"
format_stamp: "Formatted at 2026-08-09 23:08:55 on dist-test-slave-j2vl"
server_key: "85f0abf670e10928c18e38b952a49a89"
server_key_iv: "fc28affb1122df3d7388d5a70cee0ae5"
server_key_version: "encryptionkey@0"
I20260809 23:08:55.823807 12210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:55.842133 12210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:55.844640 12210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:55.844774 12210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:55.845480 12210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:55.845531 12210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:55.845558 12210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:55.845572 12210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:08:55 dist-test-slave-j2vl krb5kdc[11736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:55.853948 12210 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:41165
I20260809 23:08:55.854051 12433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:41165 every 8 connection(s)
I20260809 23:08:55.854439 12210 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:08:55.858708 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.846218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:55115 (local address 127.8.141.254:33861)
0809 23:08:55.846277 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:55.846279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:55.846515 (+ 236us) server_negotiation.cc:407] Connection header received
0809 23:08:55.847405 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:55.847407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:55.847424 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:55.847461 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:55.848657 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.849507 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.851426 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.851765 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.851846 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.854442 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:55.854444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:55.854445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:55.854455 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:55.856510 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.857179 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.857180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.857180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.857245 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.857754 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.857755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.857756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.857918 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:55.858095 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:55.858529 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 23:08:55.858620 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:08:55.858744 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.845985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33861 (local address 127.8.141.194:55115)
0809 23:08:55.846334 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:55.846400 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:55.847227 (+ 827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:55.847684 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:55.847688 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:55.848212 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:55.848557 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.848564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.849639 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.849641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.851300 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.851306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.851855 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.851857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.852010 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.852604 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:55.852614 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:55.854328 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:55.856671 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.856682 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.856685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.857069 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.857528 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.857529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.857529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.857661 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.858119 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:55.858126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:55.858274 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 23:08:55.858387 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 23:08:55.858546 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":176,"threads_started":1}
I20260809 23:08:55.859462 12434 heartbeater.cc:347] Connected to a master server at 127.8.141.254:33861
I20260809 23:08:55.859542 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12210
I20260809 23:08:55.859586 12434 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:55.859627 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:08:55.859838 8759 external_mini_cluster.cc:1428] Setting key afda81dc5acb2302eba41293788eb0a3
I20260809 23:08:55.859854 12434 heartbeater.cc:511] Master 127.8.141.254:33861 requested a full tablet report, sending...
I20260809 23:08:55.860445 11818 ts_manager.cc:194] Registered new tserver with Master: f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165)
I20260809 23:08:55.860968 11818 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:55115
I20260809 23:08:55.868418 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.860983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33861 (local address 127.8.141.194:35111)
0809 23:08:55.861213 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:55.861226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:55.861317 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:55.861517 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:55.861518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:55.861781 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:55.861856 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.861859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.862611 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.862612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.863334 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:55.863338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.863902 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.863904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:55.863974 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.864548 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:55.864553 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:55.864882 (+ 329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:55.867116 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.867117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.867117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.867328 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.867602 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:55.867607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:55.867607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:55.867672 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:55.868112 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:55.868113 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:55.868165 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 23:08:55.868226 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:08:55.868329 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":91,"threads_started":1}
I20260809 23:08:55.868546 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:55.861022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:35111 (local address 127.8.141.254:33861)
0809 23:08:55.861062 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:55.861064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:55.861277 (+ 213us) server_negotiation.cc:407] Connection header received
0809 23:08:55.861416 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:55.861417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:55.861429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:55.861501 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:55.861968 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.862430 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.863437 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:55.863768 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:55.863852 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:55.864983 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:55.864985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:55.864985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:55.864994 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:55.866982 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.867419 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.867421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.867421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.867502 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:55.867745 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:55.867746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:55.867747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:55.867995 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:55.868081 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:55.868286 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 23:08:55.868388 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:08:55.898456 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:33861
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:08:55.992662 12453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:08:55.992923 12453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:08:55.992956 12453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:08:55.994292 12453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:08:55.994347 12453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:08:55.994371 12453 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 23:08:55.994390 12453 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:08:55.994433 12453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:08:55.996057 12453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:33861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:08:55.996286 12453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:08:55.996502 12453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:08:56.002753 12465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:56.002753 12461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:08:56.002746 12462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:08:56.003592 12453 server_base.cc:1034] running on GCE node
I20260809 23:08:56.003870 12453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:08:56.004204 12453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:08:56.005532 12453 hybrid_clock.cc:648] HybridClock initialized: now 1786316936005472 us; error 73 us; skew 500 ppm
Aug 09 23:08:56 dist-test-slave-j2vl krb5kdc[11736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:08:56.008496 12453 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:08:56.009022 12453 webserver.cc:460] Webserver started at http://127.8.141.195:42855/ using document root <none> and password file <none>
I20260809 23:08:56.009270 12453 fs_manager.cc:359] Metadata directory not provided
I20260809 23:08:56.009343 12453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:08:56.009533 12453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:08:56.010910 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "7e152f670aad45d4bf96014a196ba45e"
format_stamp: "Formatted at 2026-08-09 23:08:56 on dist-test-slave-j2vl"
server_key: "f4d1caab9cd588a0100cbee2a80d06b7"
server_key_iv: "619b35d66b7a757305a0fb8a0be87fcb"
server_key_version: "encryptionkey@0"
I20260809 23:08:56.011350 12453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "7e152f670aad45d4bf96014a196ba45e"
format_stamp: "Formatted at 2026-08-09 23:08:56 on dist-test-slave-j2vl"
server_key: "f4d1caab9cd588a0100cbee2a80d06b7"
server_key_iv: "619b35d66b7a757305a0fb8a0be87fcb"
server_key_version: "encryptionkey@0"
I20260809 23:08:56.012861 12453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:08:56.013950 12478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:56.014369 12453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:08:56.014518 12453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "7e152f670aad45d4bf96014a196ba45e"
format_stamp: "Formatted at 2026-08-09 23:08:56 on dist-test-slave-j2vl"
server_key: "f4d1caab9cd588a0100cbee2a80d06b7"
server_key_iv: "619b35d66b7a757305a0fb8a0be87fcb"
server_key_version: "encryptionkey@0"
I20260809 23:08:56.014613 12453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:08:56.023267 12453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:08:56.025877 12453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:08:56.026049 12453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:08:56.026633 12453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:08:56.026703 12453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:56.026763 12453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:08:56.026800 12453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:08:56 dist-test-slave-j2vl krb5kdc[11736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:56.034837 12453 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:37663
I20260809 23:08:56.034976 12661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:37663 every 8 connection(s)
I20260809 23:08:56.035279 12453 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:08:56.038141 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.027326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:40349 (local address 127.8.141.254:33861)
0809 23:08:56.027384 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.027386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.027588 (+ 202us) server_negotiation.cc:407] Connection header received
0809 23:08:56.028298 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.028300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.028314 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.028405 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:56.029289 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.030174 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.031211 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.031530 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.031610 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.034937 (+ 3327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:56.034939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:56.034940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:56.034950 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:56.036758 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.037170 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.037171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.037171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.037226 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.037533 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.037534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.037534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.037649 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:56.037706 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.037997 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.038070 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:08:56.038268 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.027273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33861 (local address 127.8.141.195:40349)
0809 23:08:56.027470 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.027512 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.028171 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.028502 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.028507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.028888 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:56.029179 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.029186 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.030294 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.030295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.031109 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.031111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.031636 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.031638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.031798 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.032423 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:56.032434 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:56.034809 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:56.036848 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:56.036855 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:56.036857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:56.037097 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:56.037290 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:56.037291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:56.037291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:56.037473 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:56.037709 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:56.037715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:56.037842 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.037934 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.038075 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
I20260809 23:08:56.038800 12664 heartbeater.cc:347] Connected to a master server at 127.8.141.254:33861
I20260809 23:08:56.038884 12664 heartbeater.cc:464] Registering TS with master...
I20260809 23:08:56.039103 12664 heartbeater.cc:511] Master 127.8.141.254:33861 requested a full tablet report, sending...
I20260809 23:08:56.039520 11817 ts_manager.cc:194] Registered new tserver with Master: 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663)
I20260809 23:08:56.039932 11817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:40349
I20260809 23:08:56.045333 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12453
I20260809 23:08:56.045490 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:08:56.045696 8759 external_mini_cluster.cc:1428] Setting key defbe081b6ffa28a3a2694c882272c9d
I20260809 23:08:56.046008 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.040050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:47977 (local address 127.8.141.254:33861)
0809 23:08:56.040093 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.040095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.040270 (+ 175us) server_negotiation.cc:407] Connection header received
0809 23:08:56.040344 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.040345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.040355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.040408 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:56.040764 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.041243 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.041997 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.042195 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.042282 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.043283 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:56.043285 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:56.043285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:56.043292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:56.044585 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.044966 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.044967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.044967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.045029 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.045259 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.045260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.045260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.045464 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:56.045557 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.045754 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.045847 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:08:56.046054 12675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.039975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33861 (local address 127.8.141.195:47977)
0809 23:08:56.040198 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.040208 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.040287 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.040417 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.040418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.040621 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:56.040689 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.040690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.041337 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.041338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.041908 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.041909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.042286 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.042287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.042362 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.042823 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:56.042827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:56.043144 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:56.044696 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:56.044697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:56.044698 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:56.044890 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:56.045105 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:56.045106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:56.045107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:56.045174 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:56.045558 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:56.045559 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:56.045615 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.045699 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.045874 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":159,"threads_started":1}
I20260809 23:08:56.046706 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:08:56 dist-test-slave-j2vl krb5kdc[11736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:08:56 dist-test-slave-j2vl krb5kdc[11736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:56.077796 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.069774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.8.141.254:33861)
0809 23:08:56.069858 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.069860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.069867 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:08:56.069892 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.069893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.069903 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.069971 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:56.070694 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.071225 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.071977 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.072183 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.072259 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.074629 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:56.074631 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:56.074631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:56.074638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:56.076213 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.076737 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.076738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.076739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.076798 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.077104 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.077105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.077106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.077242 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:56.077403 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.077637 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.077720 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:08:56.080207 11818 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:08:56.081996 11818 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:56.089262 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.084799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:37663 (local address 127.0.0.1:44786)
0809 23:08:56.085324 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.085334 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.085414 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.085699 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.085700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.085702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:56.085746 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.085748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.086926 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.086928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.088023 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.088026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.088905 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.088906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.088988 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.089002 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.089080 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.089155 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":42,"thread_start_us":109,"threads_started":1}
I20260809 23:08:56.089270 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.085100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44786 (local address 127.8.141.195:37663)
0809 23:08:56.085480 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.085485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.085498 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:08:56.085545 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.085547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.085562 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.085635 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:56.085824 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.086840 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.088116 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.088806 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.088885 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.088949 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.089112 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.089163 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":113,"threads_started":1}
I20260809 23:08:56.089533 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.085170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36694 (local address 127.8.141.193:37167)
0809 23:08:56.085545 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.085549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.085559 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:08:56.085603 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.085605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.085621 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.085767 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:56.085980 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.086788 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.088286 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.088827 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.088932 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.088998 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.089301 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.089350 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":71,"threads_started":1}
I20260809 23:08:56.089262 12697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.084799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37167 (local address 127.0.0.1:36694)
0809 23:08:56.085310 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.085322 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.085414 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.085843 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.085844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.085846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:56.085895 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.085896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.086897 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.086898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.088185 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.088188 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.089031 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.089032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.089101 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.089104 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.089149 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.089181 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":72,"thread_start_us":217,"threads_started":1}
I20260809 23:08:56.089709 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.084849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54184 (local address 127.8.141.194:41165)
0809 23:08:56.085065 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.085070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.085081 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:08:56.085125 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.085127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.085142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.085239 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:56.085441 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.086703 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.088399 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.089135 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.089200 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.089254 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.089575 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.089618 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
I20260809 23:08:56.090454 12076 tablet_service.cc:1467] Processing CreateTablet for tablet 26f74b150ba24cf895aa9aa4cdac5b98 (DEFAULT_TABLE table=test-table [id=41a94880d2154a4f89a653ec42048712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:56.090878 12076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f74b150ba24cf895aa9aa4cdac5b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.090835 12561 tablet_service.cc:1467] Processing CreateTablet for tablet 26f74b150ba24cf895aa9aa4cdac5b98 (DEFAULT_TABLE table=test-table [id=41a94880d2154a4f89a653ec42048712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:56.091147 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f74b150ba24cf895aa9aa4cdac5b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.093546 12694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.084739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41165 (local address 127.0.0.1:54184)
0809 23:08:56.084924 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.084965 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.085037 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.085247 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.085250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.085252 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:56.085331 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.085333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.086803 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.086805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.088301 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.088303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.089293 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.089294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.089383 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.089388 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.089444 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.089512 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":126,"thread_start_us":71,"threads_started":1}
I20260809 23:08:56.094287 12322 tablet_service.cc:1467] Processing CreateTablet for tablet 26f74b150ba24cf895aa9aa4cdac5b98 (DEFAULT_TABLE table=test-table [id=41a94880d2154a4f89a653ec42048712]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:56.094668 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26f74b150ba24cf895aa9aa4cdac5b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.094831 12705 tablet_bootstrap.cc:492] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Bootstrap starting.
I20260809 23:08:56.095111 12707 tablet_bootstrap.cc:492] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Bootstrap starting.
I20260809 23:08:56.095692 12707 tablet_bootstrap.cc:654] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.095861 12705 tablet_bootstrap.cc:654] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.096093 12707 log.cc:824] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:56.096230 12705 log.cc:824] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:56.096972 12707 tablet_bootstrap.cc:492] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: No bootstrap required, opened a new log
I20260809 23:08:56.097047 12707 ts_tablet_manager.cc:1397] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:08:56.097129 12705 tablet_bootstrap.cc:492] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: No bootstrap required, opened a new log
I20260809 23:08:56.097205 12705 ts_tablet_manager.cc:1397] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:08:56.098820 12705 raft_consensus.cc:348] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.099031 12705 raft_consensus.cc:374] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.099081 12705 raft_consensus.cc:729] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e152f670aad45d4bf96014a196ba45e, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.099007 12710 tablet_bootstrap.cc:492] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Bootstrap starting.
I20260809 23:08:56.099220 12705 consensus_queue.cc:260] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.099350 12707 raft_consensus.cc:348] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.099511 12707 raft_consensus.cc:374] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.099534 12705 ts_tablet_manager.cc:1428] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:08:56.099556 12707 raft_consensus.cc:729] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8240c635f2141d4a86d1ee63c0528b1, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.099649 12664 heartbeater.cc:503] Master 127.8.141.254:33861 was elected leader, sending a full tablet report...
I20260809 23:08:56.099675 12707 consensus_queue.cc:260] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.099959 12710 tablet_bootstrap.cc:654] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.099954 12707 ts_tablet_manager.cc:1428] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:08:56.100004 12196 heartbeater.cc:503] Master 127.8.141.254:33861 was elected leader, sending a full tablet report...
I20260809 23:08:56.100342 12710 log.cc:824] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Log is configured to *not* fsync() on all Append() calls
I20260809 23:08:56.101122 12710 tablet_bootstrap.cc:492] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: No bootstrap required, opened a new log
I20260809 23:08:56.101213 12710 ts_tablet_manager.cc:1397] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:08:56.102924 12710 raft_consensus.cc:348] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.103073 12710 raft_consensus.cc:374] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.103120 12710 raft_consensus.cc:729] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.103271 12710 consensus_queue.cc:260] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.103543 12710 ts_tablet_manager.cc:1428] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:08:56.103626 12434 heartbeater.cc:503] Master 127.8.141.254:33861 was elected leader, sending a full tablet report...
W20260809 23:08:56.105163 12436 tablet.cc:2369] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:08:56.167294 12200 tablet.cc:2369] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:08:56.181439 12718 raft_consensus.cc:482] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:56.181607 12718 raft_consensus.cc:504] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.182013 12718 leader_election.cc:288] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663), d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167)
I20260809 23:08:56.184794 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.182263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:57925 (local address 127.8.141.193:37167)
0809 23:08:56.182362 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.182365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.182377 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:08:56.182409 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.182410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.182421 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.182498 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:56.182654 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.183155 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.184078 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.184441 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.184527 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.184537 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.184686 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.184723 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 23:08:56.184794 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.182234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37167 (local address 127.8.141.194:57925)
0809 23:08:56.182291 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.182304 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.182351 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.182505 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.182506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.182507 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:56.182584 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.182587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.183247 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.183248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.183985 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.183987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.184527 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.184528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.184612 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.184615 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.184681 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.184722 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:08:56.184971 12702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.182281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:55579 (local address 127.8.141.195:37663)
0809 23:08:56.182354 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.182357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.182560 (+ 203us) server_negotiation.cc:407] Connection header received
0809 23:08:56.182630 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.182631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.182640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.182714 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:56.182873 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.183371 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.184259 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.184621 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.184707 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.184715 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.184861 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.184894 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 23:08:56.184993 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.182241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:37663 (local address 127.8.141.194:55579)
0809 23:08:56.182488 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.182497 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.182573 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.182717 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.182718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.182720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:56.182797 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.182798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.183468 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.183469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.184168 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.184170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.184707 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.184708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.184786 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.184789 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.184856 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.184892 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":57,"thread_start_us":121,"threads_started":1}
I20260809 23:08:56.185153 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f74b150ba24cf895aa9aa4cdac5b98" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1" is_pre_election: true
I20260809 23:08:56.185316 12102 raft_consensus.cc:2454] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 0.
I20260809 23:08:56.185298 12597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f74b150ba24cf895aa9aa4cdac5b98" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e152f670aad45d4bf96014a196ba45e" is_pre_election: true
I20260809 23:08:56.185464 12597 raft_consensus.cc:2454] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 0.
I20260809 23:08:56.185621 12252 leader_election.cc:302] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8240c635f2141d4a86d1ee63c0528b1, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:56.185748 12718 raft_consensus.cc:2781] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:56.185832 12718 raft_consensus.cc:482] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:56.185868 12718 raft_consensus.cc:3037] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.187011 12718 raft_consensus.cc:504] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.187168 12718 leader_election.cc:288] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 election: Requested vote from peers 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663), d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167)
I20260809 23:08:56.187325 12597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f74b150ba24cf895aa9aa4cdac5b98" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e152f670aad45d4bf96014a196ba45e"
I20260809 23:08:56.187348 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26f74b150ba24cf895aa9aa4cdac5b98" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1"
I20260809 23:08:56.187422 12597 raft_consensus.cc:3037] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.187423 12102 raft_consensus.cc:3037] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.188167 12102 raft_consensus.cc:2454] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 1.
I20260809 23:08:56.188172 12597 raft_consensus.cc:2454] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 1.
I20260809 23:08:56.188360 12252 leader_election.cc:302] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8240c635f2141d4a86d1ee63c0528b1, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:56.188472 12718 raft_consensus.cc:2781] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:56.188643 12718 raft_consensus.cc:686] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 LEADER]: Becoming Leader. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Running, Role: LEADER
I20260809 23:08:56.188757 12718 consensus_queue.cc:237] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } }
I20260809 23:08:56.189777 11817 catalog_manager.cc:5495] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be reported cstate change: term changed from 0 to 1, leader changed from <none> to f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194). New cstate: current_term: 1 leader_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } health_report { overall_health: UNKNOWN } } }
Aug 09 23:08:56 dist-test-slave-j2vl krb5kdc[11736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316935, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:56.246263 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.238253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33861 (local address 127.0.0.1:39662)
0809 23:08:56.238527 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.238537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.238655 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.239041 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.239042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.239372 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:08:56.239424 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.239426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.240207 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.240208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.240769 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.240771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.241172 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.241173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.241224 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.241881 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:08:56.241886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:08:56.243414 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:08:56.244992 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:56.245000 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:56.245002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:56.245205 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:56.245536 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:08:56.245537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:08:56.245538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:08:56.245607 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:08:56.245917 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:08:56.245924 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:08:56.245975 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.246050 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.246157 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":136,"threads_started":1}
I20260809 23:08:56.246321 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.238357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39662 (local address 127.8.141.254:33861)
0809 23:08:56.238484 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.238487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.238605 (+ 118us) server_negotiation.cc:407] Connection header received
0809 23:08:56.238751 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.238752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.238762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.238966 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:56.239521 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.240110 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.240865 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.241048 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.241125 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.243514 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:56.243516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:56.243516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:56.243523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:56.244896 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.245295 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.245296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.245297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.245453 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:56.245689 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:56.245690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:56.245690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:56.245837 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:56.245894 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.246080 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.246211 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":91}
I20260809 23:08:56.248873 11817 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:39662:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 23:08:56.249078 11817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:56.250802 12322 tablet_service.cc:1467] Processing CreateTablet for tablet 3478f9351a544be48356046d9474120e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5197c9ccd30c446aa87f2e26a7264dde]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:08:56.250820 12076 tablet_service.cc:1467] Processing CreateTablet for tablet 3478f9351a544be48356046d9474120e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5197c9ccd30c446aa87f2e26a7264dde]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:08:56.250849 12561 tablet_service.cc:1467] Processing CreateTablet for tablet 3478f9351a544be48356046d9474120e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5197c9ccd30c446aa87f2e26a7264dde]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:08:56.251004 12076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3478f9351a544be48356046d9474120e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.251004 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3478f9351a544be48356046d9474120e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.251045 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3478f9351a544be48356046d9474120e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.252373 12705 tablet_bootstrap.cc:492] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: Bootstrap starting.
I20260809 23:08:56.252542 12707 tablet_bootstrap.cc:492] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: Bootstrap starting.
I20260809 23:08:56.252542 12710 tablet_bootstrap.cc:492] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: Bootstrap starting.
I20260809 23:08:56.252986 12710 tablet_bootstrap.cc:654] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.252986 12707 tablet_bootstrap.cc:654] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.253067 12705 tablet_bootstrap.cc:654] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.253765 12710 tablet_bootstrap.cc:492] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: No bootstrap required, opened a new log
I20260809 23:08:56.253770 12707 tablet_bootstrap.cc:492] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: No bootstrap required, opened a new log
I20260809 23:08:56.253839 12705 tablet_bootstrap.cc:492] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: No bootstrap required, opened a new log
I20260809 23:08:56.253837 12707 ts_tablet_manager.cc:1397] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:56.253840 12710 ts_tablet_manager.cc:1397] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:56.253893 12705 ts_tablet_manager.cc:1397] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:08:56.254002 12710 raft_consensus.cc:348] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.254037 12707 raft_consensus.cc:348] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.254101 12710 raft_consensus.cc:374] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.254060 12705 raft_consensus.cc:348] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.254110 12707 raft_consensus.cc:374] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.254132 12710 raft_consensus.cc:729] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.254133 12705 raft_consensus.cc:374] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.254138 12707 raft_consensus.cc:729] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8240c635f2141d4a86d1ee63c0528b1, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.254160 12705 raft_consensus.cc:729] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e152f670aad45d4bf96014a196ba45e, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.254175 12710 consensus_queue.cc:260] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.254190 12707 consensus_queue.cc:260] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.254210 12705 consensus_queue.cc:260] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.254290 12710 ts_tablet_manager.cc:1428] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:56.254298 12707 ts_tablet_manager.cc:1428] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:08:56.254298 12705 ts_tablet_manager.cc:1428] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:56.254376 12712 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.254438 12718 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.254449 12713 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.254509 12713 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:56.254503 12712 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:56.254511 12718 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 23:08:56.285972 12666 tablet.cc:2369] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:08:56.638522 12718 consensus_queue.cc:1035] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:56.641408 12740 consensus_queue.cc:1035] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:56.654678 12712 raft_consensus.cc:482] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:56.654829 12712 raft_consensus.cc:504] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.655267 12712 leader_election.cc:288] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165)
I20260809 23:08:56.658424 12704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.655623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:58051 (local address 127.8.141.193:37167)
0809 23:08:56.655682 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.655685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.655727 (+ 42us) server_negotiation.cc:407] Connection header received
0809 23:08:56.655804 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.655805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.655814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.655890 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:56.656053 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.656662 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.657687 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.658057 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.658145 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.658155 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.658306 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.658344 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:08:56.658444 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.655578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:35009 (local address 127.8.141.194:41165)
0809 23:08:56.655732 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.655734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.655744 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:08:56.655807 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.655807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.655818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.655903 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:08:56.656057 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.656597 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.657686 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.658069 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.658158 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.658167 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.658319 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.658360 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260809 23:08:56.658447 12743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.655498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37167 (local address 127.8.141.195:58051)
0809 23:08:56.655665 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.655677 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.655745 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.655895 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.655896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.655898 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:56.655970 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.655977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.656762 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.656763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.657602 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.657604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.658144 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.658145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.658224 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.658228 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.658296 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.658344 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260809 23:08:56.658453 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.655477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41165 (local address 127.8.141.195:35009)
0809 23:08:56.655641 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.655686 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.655747 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.655905 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.655906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.655908 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:08:56.655982 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.655984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.656680 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.656681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.657596 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.657599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.658157 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.658158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.658236 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.658239 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.658300 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.658347 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":3,"thread_start_us":93,"threads_started":1}
I20260809 23:08:56.658834 12359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3478f9351a544be48356046d9474120e" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" is_pre_election: true
I20260809 23:08:56.658988 12359 raft_consensus.cc:2454] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 0.
I20260809 23:08:56.659072 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3478f9351a544be48356046d9474120e" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1" is_pre_election: true
I20260809 23:08:56.659195 12102 raft_consensus.cc:2454] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 0.
I20260809 23:08:56.659209 12484 leader_election.cc:302] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e152f670aad45d4bf96014a196ba45e, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:56.659359 12712 raft_consensus.cc:2781] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:56.659400 12712 raft_consensus.cc:482] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:56.659435 12712 raft_consensus.cc:3037] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.660286 12712 raft_consensus.cc:504] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.660478 12712 leader_election.cc:288] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 election: Requested vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165)
I20260809 23:08:56.660637 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3478f9351a544be48356046d9474120e" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1"
I20260809 23:08:56.660653 12359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3478f9351a544be48356046d9474120e" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f590b7a531b14c7aa2e4db2c7a1715be"
I20260809 23:08:56.660714 12102 raft_consensus.cc:3037] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.660738 12359 raft_consensus.cc:3037] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.661377 12102 raft_consensus.cc:2454] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 1.
I20260809 23:08:56.661638 12480 leader_election.cc:302] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e152f670aad45d4bf96014a196ba45e, d8240c635f2141d4a86d1ee63c0528b1; no voters:
I20260809 23:08:56.661643 12359 raft_consensus.cc:2454] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 1.
I20260809 23:08:56.661827 12712 raft_consensus.cc:2781] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:56.662014 12712 raft_consensus.cc:686] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [term 1 LEADER]: Becoming Leader. State: Replica: 7e152f670aad45d4bf96014a196ba45e, State: Running, Role: LEADER
I20260809 23:08:56.662117 12712 consensus_queue.cc:237] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.662518 12747 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: TxnStatusTablet state changed. Reason: New leader 7e152f670aad45d4bf96014a196ba45e. Latest consensus state: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.662645 12747 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:08:56.662917 11815 catalog_manager.cc:5495] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e152f670aad45d4bf96014a196ba45e (127.8.141.195). New cstate: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } health_report { overall_health: HEALTHY } } }
I20260809 23:08:56.663278 12749 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 23:08:56.663410 12749 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 23:08:56.698369 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.695359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:37663 (local address 127.0.0.1:44802)
0809 23:08:56.695445 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.695460 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.695615 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.695912 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.695914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.695915 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:08:56.695974 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.695982 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.696687 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.696689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.697303 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.697305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.697773 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.697774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.697851 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.697867 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.698173 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.698185 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.698237 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.698292 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 23:08:56.698436 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.695576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.8.141.195:37663)
0809 23:08:56.695720 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.695722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.695730 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:08:56.695773 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.695774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.695792 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.695904 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:56.696066 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.696593 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.697460 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.697688 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.697772 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.697969 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.698096 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.698195 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.698240 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.698263 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":78,"threads_started":1}
I20260809 23:08:56.700393 12102 raft_consensus.cc:1261] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Refusing update from remote peer 7e152f670aad45d4bf96014a196ba45e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:56.700469 12359 raft_consensus.cc:1261] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Refusing update from remote peer 7e152f670aad45d4bf96014a196ba45e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:56.700773 12712 consensus_queue.cc:1035] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:56.701082 12747 consensus_queue.cc:1035] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:56.702332 12740 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: TxnStatusTablet state changed. Reason: New leader 7e152f670aad45d4bf96014a196ba45e. Latest consensus state: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.702432 12740 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:56.703145 12713 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: TxnStatusTablet state changed. Reason: New leader 7e152f670aad45d4bf96014a196ba45e. Latest consensus state: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.703258 12713 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:56.703200 12712 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.703302 12712 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:08:56.703470 12740 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.703552 12740 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P f590b7a531b14c7aa2e4db2c7a1715be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:56.703598 12737 mvcc.cc:205] Tried to move back new op lower bound from 7316754172722069504 to 7316754172568739840. Current Snapshot: MvccSnapshot[applied={T|T < 7316754172722069504}]
I20260809 23:08:56.703666 12742 mvcc.cc:205] Tried to move back new op lower bound from 7316754172722069504 to 7316754172568739840. Current Snapshot: MvccSnapshot[applied={T|T < 7316754172722069504}]
I20260809 23:08:56.705220 12747 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.705442 12747 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P 7e152f670aad45d4bf96014a196ba45e: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:08:56.706084 12739 mvcc.cc:205] Tried to move back new op lower bound from 7316754172722069504 to 7316754172568739840. Current Snapshot: MvccSnapshot[applied={T|T < 7316754172722069504}]
I20260809 23:08:56.706205 12713 tablet_replica.cc:439] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } } }
I20260809 23:08:56.706308 12713 tablet_replica.cc:442] T 3478f9351a544be48356046d9474120e P d8240c635f2141d4a86d1ee63c0528b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:08:56.710860 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.707564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54188 (local address 127.8.141.194:41165)
0809 23:08:56.707614 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.707617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.707758 (+ 141us) server_negotiation.cc:407] Connection header received
0809 23:08:56.707842 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.707843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.707861 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.707920 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:56.708079 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.708641 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.709955 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.710163 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.710226 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.710470 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.710577 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.710652 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.710722 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.710745 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:08:56.717298 12770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.714033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:37663 (local address 127.8.141.194:45317)
0809 23:08:56.714230 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.714243 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.714313 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.714446 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.714447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.714449 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:08:56.714522 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.714524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.715188 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.715190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.716228 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.716231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.716649 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.716650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.716730 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.716759 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.717042 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.717044 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.717099 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.717168 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260809 23:08:56.717316 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.714144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:45317 (local address 127.8.141.195:37663)
0809 23:08:56.714213 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.714216 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.714311 (+ 95us) server_negotiation.cc:407] Connection header received
0809 23:08:56.714372 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.714373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.714382 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.714432 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:56.714612 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.715101 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.716370 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.716564 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.716627 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.716895 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.716929 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.717026 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.717150 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.717176 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:08:56.729044 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.725687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:42615 (local address 127.8.141.194:41165)
0809 23:08:56.725737 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.725739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.725880 (+ 141us) server_negotiation.cc:407] Connection header received
0809 23:08:56.725951 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.725952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.725967 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.726018 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:56.726249 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.726833 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.728155 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.728350 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.728456 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.728723 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.728768 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.728853 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.728940 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.728968 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:08:56.729097 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.725512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41165 (local address 127.8.141.195:42615)
0809 23:08:56.725812 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.725825 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.725889 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.726069 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.726071 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.726072 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:08:56.726150 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.726152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.726921 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.726922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.728059 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.728063 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.728481 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.728482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.728558 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.728575 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.728881 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.728884 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.728938 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.728989 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":109,"threads_started":1}
I20260809 23:08:56.739923 11815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:56.740072 11815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:56.741024 11815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Sending DeleteTablet for 3 replicas of tablet 26f74b150ba24cf895aa9aa4cdac5b98
I20260809 23:08:56.741448 12322 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f74b150ba24cf895aa9aa4cdac5b98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:56 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:54184
I20260809 23:08:56.741715 12076 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f74b150ba24cf895aa9aa4cdac5b98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:56 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260809 23:08:56.742108 12561 tablet_service.cc:1514] Processing DeleteTablet for tablet 26f74b150ba24cf895aa9aa4cdac5b98 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:56 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:44786
I20260809 23:08:56.742168 12780 tablet_replica.cc:330] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: stopping tablet replica
I20260809 23:08:56.742412 12780 raft_consensus.cc:2229] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:56.742637 12778 tablet_replica.cc:330] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: stopping tablet replica
I20260809 23:08:56.742604 11815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 23:08:56.742755 12778 raft_consensus.cc:2229] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 LEADER]: Raft consensus shutting down.
W20260809 23:08:56.742764 11815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:56.742857 12778 raft_consensus.cc:2258] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:56.743010 12781 tablet_replica.cc:330] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: stopping tablet replica
I20260809 23:08:56.743112 12781 raft_consensus.cc:2229] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:56.743162 12781 raft_consensus.cc:2258] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:56.743607 12781 ts_tablet_manager.cc:1905] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:56.744890 12322 tablet_service.cc:1467] Processing CreateTablet for tablet b308bc44fd6d4b949b99fcba3cd60395 (DEFAULT_TABLE table=test-table [id=3085d362c8e842fbb1d862c1a89880db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:56.745028 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b308bc44fd6d4b949b99fcba3cd60395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.745332 12778 ts_tablet_manager.cc:1905] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:56.745494 12561 tablet_service.cc:1467] Processing CreateTablet for tablet b308bc44fd6d4b949b99fcba3cd60395 (DEFAULT_TABLE table=test-table [id=3085d362c8e842fbb1d862c1a89880db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:56.745434 12076 tablet_service.cc:1467] Processing CreateTablet for tablet b308bc44fd6d4b949b99fcba3cd60395 (DEFAULT_TABLE table=test-table [id=3085d362c8e842fbb1d862c1a89880db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:56.746783 12076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b308bc44fd6d4b949b99fcba3cd60395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.747112 12710 tablet_bootstrap.cc:492] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: Bootstrap starting.
I20260809 23:08:56.747558 12710 tablet_bootstrap.cc:654] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.748085 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b308bc44fd6d4b949b99fcba3cd60395. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:56.748351 12707 tablet_bootstrap.cc:492] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: Bootstrap starting.
I20260809 23:08:56.748898 12707 tablet_bootstrap.cc:654] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.749468 12705 tablet_bootstrap.cc:492] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: Bootstrap starting.
I20260809 23:08:56.749835 12705 tablet_bootstrap.cc:654] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:56.742542 12780 raft_consensus.cc:2258] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:56.750504 12780 ts_tablet_manager.cc:1905] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:56.751062 12781 ts_tablet_manager.cc:1918] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:56.751147 12781 log.cc:1196] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/26f74b150ba24cf895aa9aa4cdac5b98
I20260809 23:08:56.752027 12781 ts_tablet_manager.cc:1939] T 26f74b150ba24cf895aa9aa4cdac5b98 P 7e152f670aad45d4bf96014a196ba45e: Deleting consensus metadata
I20260809 23:08:56.752518 11794 catalog_manager.cc:4841] TS 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663): tablet 26f74b150ba24cf895aa9aa4cdac5b98 (table test-table [id=41a94880d2154a4f89a653ec42048712]) successfully deleted
I20260809 23:08:56.753214 12705 tablet_bootstrap.cc:492] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: No bootstrap required, opened a new log
I20260809 23:08:56.753276 12705 ts_tablet_manager.cc:1397] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 23:08:56.753479 12705 raft_consensus.cc:348] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.753572 12705 raft_consensus.cc:374] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.753602 12705 raft_consensus.cc:729] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e152f670aad45d4bf96014a196ba45e, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.753671 12705 consensus_queue.cc:260] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.753782 12705 ts_tablet_manager.cc:1428] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:56.754698 12710 tablet_bootstrap.cc:492] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: No bootstrap required, opened a new log
I20260809 23:08:56.754791 12710 ts_tablet_manager.cc:1397] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260809 23:08:56.754885 12707 tablet_bootstrap.cc:492] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: No bootstrap required, opened a new log
I20260809 23:08:56.754940 12707 ts_tablet_manager.cc:1397] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260809 23:08:56.754997 12710 raft_consensus.cc:348] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.755096 12710 raft_consensus.cc:374] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.755123 12710 raft_consensus.cc:729] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.755110 12707 raft_consensus.cc:348] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.755195 12707 raft_consensus.cc:374] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:56.755177 12710 consensus_queue.cc:260] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.755221 12707 raft_consensus.cc:729] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8240c635f2141d4a86d1ee63c0528b1, State: Initialized, Role: FOLLOWER
I20260809 23:08:56.755273 12710 ts_tablet_manager.cc:1428] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:56.755271 12707 consensus_queue.cc:260] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.755393 12707 ts_tablet_manager.cc:1428] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:08:56.755580 12778 ts_tablet_manager.cc:1918] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:56.755710 12778 log.cc:1196] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/26f74b150ba24cf895aa9aa4cdac5b98
I20260809 23:08:56.756062 12778 ts_tablet_manager.cc:1939] T 26f74b150ba24cf895aa9aa4cdac5b98 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting consensus metadata
I20260809 23:08:56.756436 12780 ts_tablet_manager.cc:1918] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:56.756656 12780 log.cc:1196] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/26f74b150ba24cf895aa9aa4cdac5b98
I20260809 23:08:56.756783 11802 catalog_manager.cc:4841] TS f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165): tablet 26f74b150ba24cf895aa9aa4cdac5b98 (table test-table [id=41a94880d2154a4f89a653ec42048712]) successfully deleted
I20260809 23:08:56.757050 12780 ts_tablet_manager.cc:1939] T 26f74b150ba24cf895aa9aa4cdac5b98 P d8240c635f2141d4a86d1ee63c0528b1: Deleting consensus metadata
I20260809 23:08:56.757550 11796 catalog_manager.cc:4841] TS d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167): tablet 26f74b150ba24cf895aa9aa4cdac5b98 (table test-table [id=41a94880d2154a4f89a653ec42048712]) successfully deleted
I20260809 23:08:56.814235 12760 raft_consensus.cc:482] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:56.814369 12760 raft_consensus.cc:504] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.814551 12760 leader_election.cc:288] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165)
I20260809 23:08:56.814788 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308bc44fd6d4b949b99fcba3cd60395" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1" is_pre_election: true
I20260809 23:08:56.814836 12359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308bc44fd6d4b949b99fcba3cd60395" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" is_pre_election: true
I20260809 23:08:56.814891 12102 raft_consensus.cc:2454] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 0.
I20260809 23:08:56.814924 12359 raft_consensus.cc:2454] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 0.
I20260809 23:08:56.815096 12484 leader_election.cc:302] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e152f670aad45d4bf96014a196ba45e, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:56.815196 12760 raft_consensus.cc:2781] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:56.815231 12760 raft_consensus.cc:482] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:56.815265 12760 raft_consensus.cc:3037] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.816272 12760 raft_consensus.cc:504] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.816421 12760 leader_election.cc:288] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 election: Requested vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165)
I20260809 23:08:56.816627 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308bc44fd6d4b949b99fcba3cd60395" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1"
I20260809 23:08:56.816704 12102 raft_consensus.cc:3037] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.816740 12359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308bc44fd6d4b949b99fcba3cd60395" candidate_uuid: "7e152f670aad45d4bf96014a196ba45e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f590b7a531b14c7aa2e4db2c7a1715be"
I20260809 23:08:56.816823 12359 raft_consensus.cc:3037] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:56.817560 12102 raft_consensus.cc:2454] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 1.
I20260809 23:08:56.817776 12480 leader_election.cc:302] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e152f670aad45d4bf96014a196ba45e, d8240c635f2141d4a86d1ee63c0528b1; no voters:
I20260809 23:08:56.817795 12359 raft_consensus.cc:2454] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e152f670aad45d4bf96014a196ba45e in term 1.
I20260809 23:08:56.817854 12760 raft_consensus.cc:2781] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:56.817888 12760 raft_consensus.cc:686] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 1 LEADER]: Becoming Leader. State: Replica: 7e152f670aad45d4bf96014a196ba45e, State: Running, Role: LEADER
I20260809 23:08:56.817935 12760 consensus_queue.cc:237] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:56.818509 11818 catalog_manager.cc:5495] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e152f670aad45d4bf96014a196ba45e (127.8.141.195). New cstate: current_term: 1 leader_uuid: "7e152f670aad45d4bf96014a196ba45e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } health_report { overall_health: HEALTHY } } }
I20260809 23:08:56.829200 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.825995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.8.141.195:37663)
0809 23:08:56.826065 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.826067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.826077 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:08:56.826105 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.826105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.826116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.826182 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:56.826334 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.827039 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.828334 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.828648 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.828729 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.828889 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.828929 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.828978 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.829082 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.829107 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 23:08:56.834729 12772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.831722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:37663 (local address 127.8.141.195:57703)
0809 23:08:56.831776 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:08:56.831787 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:08:56.831831 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:08:56.832077 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:08:56.832078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:08:56.832079 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:08:56.832120 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.832122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.832984 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.832985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.833843 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:08:56.833846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.834328 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.834329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:08:56.834382 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.834388 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.834574 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.834575 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:08:56.834611 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 23:08:56.834648 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 23:08:56.834739 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:56.831895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:57703 (local address 127.8.141.195:37663)
0809 23:08:56.831957 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:56.831959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:56.831968 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:08:56.831989 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:56.831990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:56.832000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:56.832062 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:56.832188 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.832890 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.833938 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:56.834226 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:56.834316 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:56.834473 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:56.834508 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:56.834567 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:56.834615 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 23:08:56.834637 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:08:56.836920 12359 raft_consensus.cc:1261] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Refusing update from remote peer 7e152f670aad45d4bf96014a196ba45e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:56.836920 12102 raft_consensus.cc:1261] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Refusing update from remote peer 7e152f670aad45d4bf96014a196ba45e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:56.837146 12760 consensus_queue.cc:1035] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:56.837263 12747 consensus_queue.cc:1035] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:08:56.842648 12742 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 23:08:58.364118 11815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:58.364243 11815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:58.365182 11815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Sending DeleteTablet for 3 replicas of tablet b308bc44fd6d4b949b99fcba3cd60395
I20260809 23:08:58.365641 12558 tablet_service.cc:1514] Processing DeleteTablet for tablet b308bc44fd6d4b949b99fcba3cd60395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:58 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:44786
I20260809 23:08:58.365633 12322 tablet_service.cc:1514] Processing DeleteTablet for tablet b308bc44fd6d4b949b99fcba3cd60395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:58 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:54184
I20260809 23:08:58.365988 12076 tablet_service.cc:1514] Processing DeleteTablet for tablet b308bc44fd6d4b949b99fcba3cd60395 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:58 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260809 23:08:58.366325 12830 tablet_replica.cc:330] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: stopping tablet replica
I20260809 23:08:58.366330 12829 tablet_replica.cc:330] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: stopping tablet replica
I20260809 23:08:58.366482 12830 raft_consensus.cc:2229] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:58.366482 12829 raft_consensus.cc:2229] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:08:58.366660 12830 raft_consensus.cc:2258] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:58.366710 12829 raft_consensus.cc:2258] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:58.366850 12828 tablet_replica.cc:330] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: stopping tablet replica
I20260809 23:08:58.366914 12828 raft_consensus.cc:2229] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:58.366958 12828 raft_consensus.cc:2258] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:58.367185 12829 ts_tablet_manager.cc:1905] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:58.368474 12830 ts_tablet_manager.cc:1905] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:58.368924 12829 ts_tablet_manager.cc:1918] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:08:58.369001 12829 log.cc:1196] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/b308bc44fd6d4b949b99fcba3cd60395
I20260809 23:08:58.369267 12829 ts_tablet_manager.cc:1939] T b308bc44fd6d4b949b99fcba3cd60395 P 7e152f670aad45d4bf96014a196ba45e: Deleting consensus metadata
I20260809 23:08:58.369692 11794 catalog_manager.cc:4841] TS 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663): tablet b308bc44fd6d4b949b99fcba3cd60395 (table test-table [id=3085d362c8e842fbb1d862c1a89880db]) successfully deleted
I20260809 23:08:58.370714 12830 ts_tablet_manager.cc:1918] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:08:58.370853 12830 log.cc:1196] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/b308bc44fd6d4b949b99fcba3cd60395
I20260809 23:08:58.371165 12830 ts_tablet_manager.cc:1939] T b308bc44fd6d4b949b99fcba3cd60395 P d8240c635f2141d4a86d1ee63c0528b1: Deleting consensus metadata
I20260809 23:08:58.371623 11796 catalog_manager.cc:4841] TS d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167): tablet b308bc44fd6d4b949b99fcba3cd60395 (table test-table [id=3085d362c8e842fbb1d862c1a89880db]) successfully deleted
I20260809 23:08:58.372452 12828 ts_tablet_manager.cc:1905] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:58.373998 12828 ts_tablet_manager.cc:1918] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:08:58.374082 12828 log.cc:1196] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/b308bc44fd6d4b949b99fcba3cd60395
I20260809 23:08:58.374321 12828 ts_tablet_manager.cc:1939] T b308bc44fd6d4b949b99fcba3cd60395 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting consensus metadata
I20260809 23:08:58.374820 11802 catalog_manager.cc:4841] TS f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165): tablet b308bc44fd6d4b949b99fcba3cd60395 (table test-table [id=3085d362c8e842fbb1d862c1a89880db]) successfully deleted
Aug 09 23:08:58 dist-test-slave-j2vl krb5kdc[11736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316938, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:08:58 dist-test-slave-j2vl krb5kdc[11736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316938, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:08:58.397428 12843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:58.386293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39678 (local address 127.8.141.254:33861)
0809 23:08:58.386495 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:58.386498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:58.386508 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:08:58.386536 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:58.386536 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:58.386548 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:58.386628 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:08:58.387288 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:58.387806 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:58.388559 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:58.388784 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:58.388842 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:58.394734 (+ 5892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:08:58.394736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:08:58.394737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:08:58.394745 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:08:58.396201 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:58.396577 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:58.396578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:58.396578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:58.396639 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:08:58.396856 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:08:58.396857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:08:58.396857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:08:58.396982 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:08:58.397096 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:58.397197 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 23:08:58.397291 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":103,"threads_started":1}
I20260809 23:08:58.399291 11815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:08:58.399471 11815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:58.401435 12076 tablet_service.cc:1467] Processing CreateTablet for tablet 21769c61591e49d88f7b5297c64f1041 (DEFAULT_TABLE table=test-table [id=c8f8f34d8d7f4763a857fbbac585df4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:58.401461 12322 tablet_service.cc:1467] Processing CreateTablet for tablet 21769c61591e49d88f7b5297c64f1041 (DEFAULT_TABLE table=test-table [id=c8f8f34d8d7f4763a857fbbac585df4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:58.401610 12076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21769c61591e49d88f7b5297c64f1041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:58.401624 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21769c61591e49d88f7b5297c64f1041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:58.402931 12844 tablet_bootstrap.cc:492] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: Bootstrap starting.
I20260809 23:08:58.402974 12558 tablet_service.cc:1467] Processing CreateTablet for tablet 21769c61591e49d88f7b5297c64f1041 (DEFAULT_TABLE table=test-table [id=c8f8f34d8d7f4763a857fbbac585df4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:58.403110 12558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21769c61591e49d88f7b5297c64f1041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:58.403326 12844 tablet_bootstrap.cc:654] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:58.404165 12844 tablet_bootstrap.cc:492] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: No bootstrap required, opened a new log
I20260809 23:08:58.404210 12844 ts_tablet_manager.cc:1397] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:58.404567 12844 raft_consensus.cc:348] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.404630 12844 raft_consensus.cc:374] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:58.404644 12844 raft_consensus.cc:729] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8240c635f2141d4a86d1ee63c0528b1, State: Initialized, Role: FOLLOWER
I20260809 23:08:58.404667 12844 consensus_queue.cc:260] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.404803 12850 tablet_bootstrap.cc:492] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: Bootstrap starting.
I20260809 23:08:58.404877 12844 ts_tablet_manager.cc:1428] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:58.404974 12846 tablet_bootstrap.cc:492] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: Bootstrap starting.
I20260809 23:08:58.405256 12850 tablet_bootstrap.cc:654] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:58.405583 12846 tablet_bootstrap.cc:654] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:58.406132 12850 tablet_bootstrap.cc:492] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: No bootstrap required, opened a new log
I20260809 23:08:58.406178 12850 ts_tablet_manager.cc:1397] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:58.406219 12846 tablet_bootstrap.cc:492] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: No bootstrap required, opened a new log
I20260809 23:08:58.406273 12846 ts_tablet_manager.cc:1397] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:58.406319 12850 raft_consensus.cc:348] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.406390 12850 raft_consensus.cc:374] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:58.406410 12850 raft_consensus.cc:729] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e152f670aad45d4bf96014a196ba45e, State: Initialized, Role: FOLLOWER
I20260809 23:08:58.406405 12846 raft_consensus.cc:348] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.406492 12846 raft_consensus.cc:374] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:58.406467 12850 consensus_queue.cc:260] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.406513 12846 raft_consensus.cc:729] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Initialized, Role: FOLLOWER
I20260809 23:08:58.406569 12850 ts_tablet_manager.cc:1428] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:58.406581 12846 consensus_queue.cc:260] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.406760 12846 ts_tablet_manager.cc:1428] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:08:58.461051 12860 raft_consensus.cc:482] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:58.461165 12860 raft_consensus.cc:504] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.461401 12860 leader_election.cc:288] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663)
I20260809 23:08:58.461683 12597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21769c61591e49d88f7b5297c64f1041" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e152f670aad45d4bf96014a196ba45e" is_pre_election: true
I20260809 23:08:58.461798 12597 raft_consensus.cc:2454] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 0.
I20260809 23:08:58.461998 12246 leader_election.cc:302] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e152f670aad45d4bf96014a196ba45e, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:58.462109 12860 raft_consensus.cc:2781] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:58.462134 12860 raft_consensus.cc:482] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:58.462155 12860 raft_consensus.cc:3037] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:58.462031 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21769c61591e49d88f7b5297c64f1041" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1" is_pre_election: true
I20260809 23:08:58.462194 12102 raft_consensus.cc:2454] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 0.
I20260809 23:08:58.463304 12860 raft_consensus.cc:504] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.463562 12860 leader_election.cc:288] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 election: Requested vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663)
I20260809 23:08:58.463665 12597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21769c61591e49d88f7b5297c64f1041" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e152f670aad45d4bf96014a196ba45e"
I20260809 23:08:58.463693 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21769c61591e49d88f7b5297c64f1041" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1"
I20260809 23:08:58.463760 12597 raft_consensus.cc:3037] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:58.463769 12102 raft_consensus.cc:3037] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:58.464766 12597 raft_consensus.cc:2454] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 1.
I20260809 23:08:58.464774 12102 raft_consensus.cc:2454] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 1.
I20260809 23:08:58.465121 12246 leader_election.cc:302] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e152f670aad45d4bf96014a196ba45e, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:58.465224 12860 raft_consensus.cc:2781] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:58.465425 12860 raft_consensus.cc:686] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 LEADER]: Becoming Leader. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Running, Role: LEADER
I20260809 23:08:58.465494 12860 consensus_queue.cc:237] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.466292 11818 catalog_manager.cc:5495] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be reported cstate change: term changed from 0 to 1, leader changed from <none> to f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194). New cstate: current_term: 1 leader_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } health_report { overall_health: UNKNOWN } } }
I20260809 23:08:58.484002 12865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:08:58.481328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.8.141.194:41165)
0809 23:08:58.481531 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 23:08:58.481533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:08:58.481539 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:08:58.481559 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:08:58.481560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:08:58.481570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:08:58.481645 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:08:58.481808 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:58.482313 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:58.483264 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:08:58.483476 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:08:58.483553 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:08:58.483710 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:08:58.483753 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:08:58.483808 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:08:58.483877 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 23:08:58.483901 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":126,"threads_started":1}
I20260809 23:08:58.486768 12597 raft_consensus.cc:1261] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Refusing update from remote peer f590b7a531b14c7aa2e4db2c7a1715be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:58.486768 12102 raft_consensus.cc:1261] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Refusing update from remote peer f590b7a531b14c7aa2e4db2c7a1715be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:58.487010 12861 consensus_queue.cc:1035] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:58.487151 12860 consensus_queue.cc:1035] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:58.497535 11815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:58.497648 11815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:08:58.498400 11815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Sending DeleteTablet for 3 replicas of tablet 21769c61591e49d88f7b5297c64f1041
I20260809 23:08:58.498699 12322 tablet_service.cc:1514] Processing DeleteTablet for tablet 21769c61591e49d88f7b5297c64f1041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:58 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:54184
I20260809 23:08:58.498714 12076 tablet_service.cc:1514] Processing DeleteTablet for tablet 21769c61591e49d88f7b5297c64f1041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:58 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260809 23:08:58.498862 12830 tablet_replica.cc:330] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: stopping tablet replica
I20260809 23:08:58.498850 12828 tablet_replica.cc:330] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: stopping tablet replica
I20260809 23:08:58.498924 12830 raft_consensus.cc:2229] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:58.498925 12828 raft_consensus.cc:2229] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:08:58.498970 12830 raft_consensus.cc:2258] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:58.498994 12828 raft_consensus.cc:2258] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:58.499169 11815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 23:08:58.499267 12828 ts_tablet_manager.cc:1905] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 23:08:58.499323 11815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:08:58.499646 12558 tablet_service.cc:1514] Processing DeleteTablet for tablet 21769c61591e49d88f7b5297c64f1041 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:08:58 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:44786
I20260809 23:08:58.499738 12829 tablet_replica.cc:330] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: stopping tablet replica
I20260809 23:08:58.499786 12829 raft_consensus.cc:2229] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:08:58.499835 12829 raft_consensus.cc:2258] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:08:58.500111 12829 ts_tablet_manager.cc:1905] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:58.500854 12076 tablet_service.cc:1467] Processing CreateTablet for tablet 121b9114ee1d48fe925fcf91501002c5 (DEFAULT_TABLE table=test-table [id=d4868c8674f34f63afae7ed3bc09b26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:58.501014 12076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121b9114ee1d48fe925fcf91501002c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:58.501066 12828 ts_tablet_manager.cc:1918] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:58.501123 12828 log.cc:1196] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/21769c61591e49d88f7b5297c64f1041
I20260809 23:08:58.501394 12828 ts_tablet_manager.cc:1939] T 21769c61591e49d88f7b5297c64f1041 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting consensus metadata
I20260809 23:08:58.501632 12829 ts_tablet_manager.cc:1918] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:58.501709 12829 log.cc:1196] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/21769c61591e49d88f7b5297c64f1041
I20260809 23:08:58.501928 12829 ts_tablet_manager.cc:1939] T 21769c61591e49d88f7b5297c64f1041 P 7e152f670aad45d4bf96014a196ba45e: Deleting consensus metadata
I20260809 23:08:58.502362 12844 tablet_bootstrap.cc:492] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1: Bootstrap starting.
I20260809 23:08:58.502466 12830 ts_tablet_manager.cc:1905] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:08:58.502832 12844 tablet_bootstrap.cc:654] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:58.503584 12830 ts_tablet_manager.cc:1918] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:08:58.503646 12830 log.cc:1196] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/21769c61591e49d88f7b5297c64f1041
I20260809 23:08:58.503628 11802 catalog_manager.cc:4841] TS f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165): tablet 21769c61591e49d88f7b5297c64f1041 (table test-table [id=c8f8f34d8d7f4763a857fbbac585df4b]) successfully deleted
I20260809 23:08:58.503707 12844 tablet_bootstrap.cc:492] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1: No bootstrap required, opened a new log
I20260809 23:08:58.503693 12322 tablet_service.cc:1467] Processing CreateTablet for tablet 121b9114ee1d48fe925fcf91501002c5 (DEFAULT_TABLE table=test-table [id=d4868c8674f34f63afae7ed3bc09b26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:58.503749 12844 ts_tablet_manager.cc:1397] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:58.503803 12322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121b9114ee1d48fe925fcf91501002c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:58.503870 12830 ts_tablet_manager.cc:1939] T 21769c61591e49d88f7b5297c64f1041 P d8240c635f2141d4a86d1ee63c0528b1: Deleting consensus metadata
I20260809 23:08:58.503883 12844 raft_consensus.cc:348] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.503949 12844 raft_consensus.cc:374] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:58.503964 12844 raft_consensus.cc:729] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8240c635f2141d4a86d1ee63c0528b1, State: Initialized, Role: FOLLOWER
I20260809 23:08:58.504001 12844 consensus_queue.cc:260] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.504114 12844 ts_tablet_manager.cc:1428] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:58.504328 11796 catalog_manager.cc:4841] TS d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167): tablet 21769c61591e49d88f7b5297c64f1041 (table test-table [id=c8f8f34d8d7f4763a857fbbac585df4b]) successfully deleted
I20260809 23:08:58.504886 12846 tablet_bootstrap.cc:492] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: Bootstrap starting.
I20260809 23:08:58.505035 11794 catalog_manager.cc:4841] TS 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663): tablet 21769c61591e49d88f7b5297c64f1041 (table test-table [id=c8f8f34d8d7f4763a857fbbac585df4b]) successfully deleted
I20260809 23:08:58.505151 12558 tablet_service.cc:1467] Processing CreateTablet for tablet 121b9114ee1d48fe925fcf91501002c5 (DEFAULT_TABLE table=test-table [id=d4868c8674f34f63afae7ed3bc09b26f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:08:58.505268 12558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121b9114ee1d48fe925fcf91501002c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:08:58.505265 12846 tablet_bootstrap.cc:654] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:58.505861 12846 tablet_bootstrap.cc:492] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: No bootstrap required, opened a new log
I20260809 23:08:58.505930 12846 ts_tablet_manager.cc:1397] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:08:58.506052 12846 raft_consensus.cc:348] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.506129 12846 raft_consensus.cc:374] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:58.506157 12846 raft_consensus.cc:729] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Initialized, Role: FOLLOWER
I20260809 23:08:58.506204 12846 consensus_queue.cc:260] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.506328 12846 ts_tablet_manager.cc:1428] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:58.506634 12850 tablet_bootstrap.cc:492] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: Bootstrap starting.
I20260809 23:08:58.507007 12850 tablet_bootstrap.cc:654] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: Neither blocks nor log segments found. Creating new log.
I20260809 23:08:58.507709 12850 tablet_bootstrap.cc:492] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: No bootstrap required, opened a new log
I20260809 23:08:58.507778 12850 ts_tablet_manager.cc:1397] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:08:58.507944 12850 raft_consensus.cc:348] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.508018 12850 raft_consensus.cc:374] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:08:58.508033 12850 raft_consensus.cc:729] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e152f670aad45d4bf96014a196ba45e, State: Initialized, Role: FOLLOWER
I20260809 23:08:58.508065 12850 consensus_queue.cc:260] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.508175 12850 ts_tablet_manager.cc:1428] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:08:58.914479 12860 raft_consensus.cc:482] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:08:58.914636 12860 raft_consensus.cc:504] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.914906 12860 leader_election.cc:288] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663)
I20260809 23:08:58.915154 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121b9114ee1d48fe925fcf91501002c5" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1" is_pre_election: true
I20260809 23:08:58.915202 12597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121b9114ee1d48fe925fcf91501002c5" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e152f670aad45d4bf96014a196ba45e" is_pre_election: true
I20260809 23:08:58.915297 12102 raft_consensus.cc:2454] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 0.
I20260809 23:08:58.915290 12597 raft_consensus.cc:2454] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 0.
I20260809 23:08:58.915508 12252 leader_election.cc:302] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7e152f670aad45d4bf96014a196ba45e, d8240c635f2141d4a86d1ee63c0528b1, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:58.915628 12860 raft_consensus.cc:2781] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:08:58.915705 12860 raft_consensus.cc:482] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:08:58.915737 12860 raft_consensus.cc:3037] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:58.916481 12860 raft_consensus.cc:504] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.916649 12860 leader_election.cc:288] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 election: Requested vote from peers d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167), 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663)
I20260809 23:08:58.916805 12597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121b9114ee1d48fe925fcf91501002c5" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e152f670aad45d4bf96014a196ba45e"
I20260809 23:08:58.916853 12102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121b9114ee1d48fe925fcf91501002c5" candidate_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8240c635f2141d4a86d1ee63c0528b1"
I20260809 23:08:58.916909 12597 raft_consensus.cc:3037] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:58.916914 12102 raft_consensus.cc:3037] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:08:58.917665 12102 raft_consensus.cc:2454] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 1.
I20260809 23:08:58.917773 12597 raft_consensus.cc:2454] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f590b7a531b14c7aa2e4db2c7a1715be in term 1.
I20260809 23:08:58.917906 12252 leader_election.cc:302] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8240c635f2141d4a86d1ee63c0528b1, f590b7a531b14c7aa2e4db2c7a1715be; no voters:
I20260809 23:08:58.917996 12860 raft_consensus.cc:2781] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:08:58.918056 12860 raft_consensus.cc:686] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 LEADER]: Becoming Leader. State: Replica: f590b7a531b14c7aa2e4db2c7a1715be, State: Running, Role: LEADER
I20260809 23:08:58.918118 12860 consensus_queue.cc:237] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } }
I20260809 23:08:58.918839 11816 catalog_manager.cc:5495] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be reported cstate change: term changed from 0 to 1, leader changed from <none> to f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194). New cstate: current_term: 1 leader_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f590b7a531b14c7aa2e4db2c7a1715be" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 } health_report { overall_health: UNKNOWN } } }
I20260809 23:08:58.954890 12597 raft_consensus.cc:1261] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Refusing update from remote peer f590b7a531b14c7aa2e4db2c7a1715be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:58.954906 12102 raft_consensus.cc:1261] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Refusing update from remote peer f590b7a531b14c7aa2e4db2c7a1715be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:08:58.955265 12872 consensus_queue.cc:1035] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e152f670aad45d4bf96014a196ba45e" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:08:58.955384 12860 consensus_queue.cc:1035] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8240c635f2141d4a86d1ee63c0528b1" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:08:58.960546 12805 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 23:09:00.479074 11816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:00.479187 11816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:00.480336 11816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67e9e021bec94aedb1f925c2e8c33634: Sending DeleteTablet for 3 replicas of tablet 121b9114ee1d48fe925fcf91501002c5
I20260809 23:09:00.480706 12076 tablet_service.cc:1514] Processing DeleteTablet for tablet 121b9114ee1d48fe925fcf91501002c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:00 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:36694
I20260809 23:09:00.480697 12322 tablet_service.cc:1514] Processing DeleteTablet for tablet 121b9114ee1d48fe925fcf91501002c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:00 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:54184
I20260809 23:09:00.480728 12558 tablet_service.cc:1514] Processing DeleteTablet for tablet 121b9114ee1d48fe925fcf91501002c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:00 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:44786
I20260809 23:09:00.480926 12927 tablet_replica.cc:330] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1: stopping tablet replica
I20260809 23:09:00.480953 12931 tablet_replica.cc:330] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: stopping tablet replica
I20260809 23:09:00.480957 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11956
I20260809 23:09:00.480988 12927 raft_consensus.cc:2229] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:00.481029 12927 raft_consensus.cc:2258] T 121b9114ee1d48fe925fcf91501002c5 P d8240c635f2141d4a86d1ee63c0528b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:00.487478 12928 tablet_replica.cc:330] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: stopping tablet replica
I20260809 23:09:00.487571 12928 raft_consensus.cc:2229] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:00.487649 12928 raft_consensus.cc:2258] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:00.481005 12931 raft_consensus.cc:2229] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:00.491149 12931 raft_consensus.cc:2258] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:00.491482 12928 ts_tablet_manager.cc:1905] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:00.491585 12931 ts_tablet_manager.cc:1905] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:00.493512 12928 ts_tablet_manager.cc:1918] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:09:00.493533 12931 ts_tablet_manager.cc:1918] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 23:09:00.493597 12931 log.cc:1196] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/121b9114ee1d48fe925fcf91501002c5
I20260809 23:09:00.493597 12928 log.cc:1196] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/121b9114ee1d48fe925fcf91501002c5
I20260809 23:09:00.493880 12928 ts_tablet_manager.cc:1939] T 121b9114ee1d48fe925fcf91501002c5 P f590b7a531b14c7aa2e4db2c7a1715be: Deleting consensus metadata
I20260809 23:09:00.493880 12931 ts_tablet_manager.cc:1939] T 121b9114ee1d48fe925fcf91501002c5 P 7e152f670aad45d4bf96014a196ba45e: Deleting consensus metadata
I20260809 23:09:00.494581 11802 catalog_manager.cc:4841] TS f590b7a531b14c7aa2e4db2c7a1715be (127.8.141.194:41165): tablet 121b9114ee1d48fe925fcf91501002c5 (table test-table [id=d4868c8674f34f63afae7ed3bc09b26f]) successfully deleted
I20260809 23:09:00.494596 11794 catalog_manager.cc:4841] TS 7e152f670aad45d4bf96014a196ba45e (127.8.141.195:37663): tablet 121b9114ee1d48fe925fcf91501002c5 (table test-table [id=d4868c8674f34f63afae7ed3bc09b26f]) successfully deleted
W20260809 23:09:00.503047 11796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:37167 (error 108)
I20260809 23:09:00.503767 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12210
I20260809 23:09:00.503804 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:00.503574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37167 (local address 127.0.0.1:36698)
0809 23:09:00.503720 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:00.503764 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
W20260809 23:09:00.507871 11796 catalog_manager.cc:4584] TS d8240c635f2141d4a86d1ee63c0528b1 (127.8.141.193:37167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 121b9114ee1d48fe925fcf91501002c5: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37167: connect: Connection refused (error 111)
I20260809 23:09:00.519384 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12453
I20260809 23:09:00.540298 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11761
2026-08-09T23:09:00Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6713 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:00 dist-test-slave-j2vl krb5kdc[12941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:00 dist-test-slave-j2vl krb5kdc[12941](info): set up 2 sockets
Aug 09 23:09:00 dist-test-slave-j2vl krb5kdc[12941](info): commencing operation
krb5kdc: starting...
W20260809 23:09:01.872939 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/test-user.keytab.
W20260809 23:09:02.468046 8759 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.460s user 0.000s sys 0.002s
Aug 09 23:09:02 dist-test-slave-j2vl krb5kdc[12941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
W20260809 23:09:03.494552 8759 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.141.254: real 0.999s user 0.000s sys 0.004s
I20260809 23:09:03.495054 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:38761
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:38761
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:03.601270 12963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:03.601711 12963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:03.601768 12963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:03.603253 12963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:03.603328 12963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:03.603348 12963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:03.603363 12963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:03.603377 12963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:03.605321 12963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:38761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:38761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:03.605720 12963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:03.606020 12963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:03.612725 12973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:03.612725 12977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:03.612766 12971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:03.613250 12963 server_base.cc:1034] running on GCE node
I20260809 23:09:03.613608 12963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:03.613950 12963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:03.615142 12963 hybrid_clock.cc:648] HybridClock initialized: now 1786316943615115 us; error 44 us; skew 500 ppm
Aug 09 23:09:03 dist-test-slave-j2vl krb5kdc[12941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316943, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:03.622623 12963 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:03.623216 12963 webserver.cc:460] Webserver started at http://127.8.141.254:41197/ using document root <none> and password file <none>
I20260809 23:09:03.623502 12963 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:03.623554 12963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:03.623689 12963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:03.624871 12963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "c95c2321fb5f4211aff2e97da41c55e1"
format_stamp: "Formatted at 2026-08-09 23:09:03 on dist-test-slave-j2vl"
server_key: "3a70c96c99ef2444f9666af45244e013"
server_key_iv: "9eb32b454ed72c09ff6fcbb664039637"
server_key_version: "encryptionkey@0"
I20260809 23:09:03.625272 12963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "c95c2321fb5f4211aff2e97da41c55e1"
format_stamp: "Formatted at 2026-08-09 23:09:03 on dist-test-slave-j2vl"
server_key: "3a70c96c99ef2444f9666af45244e013"
server_key_iv: "9eb32b454ed72c09ff6fcbb664039637"
server_key_version: "encryptionkey@0"
I20260809 23:09:03.627002 12963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 23:09:03.628116 12987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:03.628348 12963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:03.628494 12963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "c95c2321fb5f4211aff2e97da41c55e1"
format_stamp: "Formatted at 2026-08-09 23:09:03 on dist-test-slave-j2vl"
server_key: "3a70c96c99ef2444f9666af45244e013"
server_key_iv: "9eb32b454ed72c09ff6fcbb664039637"
server_key_version: "encryptionkey@0"
I20260809 23:09:03.628567 12963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:03.650101 12963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:03.653086 12963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:03.653239 12963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:03.658413 12963 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:38761
I20260809 23:09:03.658558 13084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:38761 every 8 connection(s)
I20260809 23:09:03.658917 12963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:03.660123 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:03.662726 13087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1: Bootstrap starting.
I20260809 23:09:03.662897 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12963
I20260809 23:09:03.662999 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:03.663177 8759 external_mini_cluster.cc:1428] Setting key 105ae346b3c50e6ed34c40de786eca39
I20260809 23:09:03.663616 13087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:03.664059 13087 log.cc:824] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:03.665036 13087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1: No bootstrap required, opened a new log
I20260809 23:09:03.667560 13087 raft_consensus.cc:348] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95c2321fb5f4211aff2e97da41c55e1" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38761 } }
I20260809 23:09:03.667778 13087 raft_consensus.cc:374] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:03.667816 13087 raft_consensus.cc:729] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c95c2321fb5f4211aff2e97da41c55e1, State: Initialized, Role: FOLLOWER
I20260809 23:09:03.667943 13087 consensus_queue.cc:260] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95c2321fb5f4211aff2e97da41c55e1" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38761 } }
I20260809 23:09:03.668020 13087 raft_consensus.cc:388] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:03.668047 13087 raft_consensus.cc:482] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:03.668093 13087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 23:09:03 dist-test-slave-j2vl krb5kdc[12941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:03.674997 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:03.663882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.8.141.254:38761)
0809 23:09:03.664083 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:03.664092 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:03.664120 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:09:03.664802 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:03.664811 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:03.664867 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:03.664998 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:03.666061 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:03.667202 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:03.668287 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:03.668727 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:03.668829 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:03.671428 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:03.671433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:03.671433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:03.671447 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:03.673235 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:03.673688 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:03.673692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:03.673692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:03.673790 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:03.674035 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:03.674036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:03.674036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:03.674303 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:03.674493 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:03.674624 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 23:09:03.674797 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":104,"threads_started":1}
I20260809 23:09:03.686311 13087 raft_consensus.cc:504] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95c2321fb5f4211aff2e97da41c55e1" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38761 } }
I20260809 23:09:03.686664 13087 leader_election.cc:302] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c95c2321fb5f4211aff2e97da41c55e1; no voters:
I20260809 23:09:03.686930 13087 leader_election.cc:288] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:03.687063 13094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:03.687252 13094 raft_consensus.cc:686] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [term 1 LEADER]: Becoming Leader. State: Replica: c95c2321fb5f4211aff2e97da41c55e1, State: Running, Role: LEADER
I20260809 23:09:03.687309 13087 sys_catalog.cc:564] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:03.687369 13094 consensus_queue.cc:237] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95c2321fb5f4211aff2e97da41c55e1" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38761 } }
I20260809 23:09:03.687799 13095 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c95c2321fb5f4211aff2e97da41c55e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95c2321fb5f4211aff2e97da41c55e1" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38761 } } }
I20260809 23:09:03.688081 13095 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:03.688007 13097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c95c2321fb5f4211aff2e97da41c55e1. Latest consensus state: current_term: 1 leader_uuid: "c95c2321fb5f4211aff2e97da41c55e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c95c2321fb5f4211aff2e97da41c55e1" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38761 } } }
I20260809 23:09:03.688366 13097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:03.688633 13111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:03.689188 13111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:03.691738 13111 catalog_manager.cc:1347] Generated new cluster ID: 7338f011e5d241129cb5572aa01b9f35
I20260809 23:09:03.691820 13111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:03.698930 13111 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:03.699527 13111 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:03.704860 13111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c95c2321fb5f4211aff2e97da41c55e1: Generated new TSK 0
I20260809 23:09:03.705128 13111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:09:03.781924 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38761
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:03.894676 13129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:03.894955 13129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:03.894986 13129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:03.896271 13129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:03.896323 13129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:03.896353 13129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:03.897990 13129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:03.898226 13129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:03.898438 13129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:03.905441 13143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:03.905467 13129 server_base.cc:1034] running on GCE node
W20260809 23:09:03.905435 13135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:03.905435 13137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:03.905979 13129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:03.906283 13129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:03.907502 13129 hybrid_clock.cc:648] HybridClock initialized: now 1786316943907481 us; error 43 us; skew 500 ppm
Aug 09 23:09:03 dist-test-slave-j2vl krb5kdc[12941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316943, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:03.910920 13129 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:09:03.911526 13129 webserver.cc:460] Webserver started at http://127.8.141.193:40069/ using document root <none> and password file <none>
I20260809 23:09:03.911819 13129 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:03.911904 13129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:03.912061 13129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:03.913194 13129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "8632c8bac103445385ef57fe236248cb"
format_stamp: "Formatted at 2026-08-09 23:09:03 on dist-test-slave-j2vl"
server_key: "d803fca763da6b0511e3070ec1d8a159"
server_key_iv: "bc7dcb96eddcf2cdb20b2fbc593cb860"
server_key_version: "encryptionkey@0"
I20260809 23:09:03.913663 13129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "8632c8bac103445385ef57fe236248cb"
format_stamp: "Formatted at 2026-08-09 23:09:03 on dist-test-slave-j2vl"
server_key: "d803fca763da6b0511e3070ec1d8a159"
server_key_iv: "bc7dcb96eddcf2cdb20b2fbc593cb860"
server_key_version: "encryptionkey@0"
I20260809 23:09:03.915406 13129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:03.916452 13155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:03.916960 13129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:03.917112 13129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "8632c8bac103445385ef57fe236248cb"
format_stamp: "Formatted at 2026-08-09 23:09:03 on dist-test-slave-j2vl"
server_key: "d803fca763da6b0511e3070ec1d8a159"
server_key_iv: "bc7dcb96eddcf2cdb20b2fbc593cb860"
server_key_version: "encryptionkey@0"
I20260809 23:09:03.917201 13129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:03.925086 13129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:03.927799 13129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:03.927985 13129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:03.928247 13129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:03.928596 13129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:03.928650 13129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:03.928695 13129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:03.928725 13129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:03.936401 13129 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:35259
I20260809 23:09:03.936673 13354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:35259 every 8 connection(s)
I20260809 23:09:03.936875 13129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:03.939597 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13129
I20260809 23:09:03.939687 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:03.939905 8759 external_mini_cluster.cc:1428] Setting key f229d68d49f0412f3bc92d24ebf28b73
Aug 09 23:09:03 dist-test-slave-j2vl krb5kdc[12941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316943, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:03.950109 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:03.937692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:54303 (local address 127.8.141.254:38761)
0809 23:09:03.937784 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:03.937786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:03.938065 (+ 279us) server_negotiation.cc:407] Connection header received
0809 23:09:03.938591 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:03.938593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:03.938612 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:03.938706 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:03.939881 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:03.940564 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:03.941738 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:03.942101 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:03.942190 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:03.946175 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:03.946178 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:03.946178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:03.946190 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:03.948064 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:03.948639 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:03.948641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:03.948641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:03.948728 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:03.949076 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:03.949077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:03.949077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:03.949256 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:03.949432 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:03.949866 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 23:09:03.949981 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 23:09:03.950197 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:03.937667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38761 (local address 127.8.141.193:54303)
0809 23:09:03.937869 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:03.937910 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:03.938438 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:03.938777 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:03.938780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:03.939425 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:03.939645 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:03.939651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:03.940709 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:03.940710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:03.941636 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:03.941638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:03.942252 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:03.942254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:03.942381 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:03.943030 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:03.943040 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:03.946038 (+ 2998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:03.948196 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:03.948210 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:03.948212 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:03.948526 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:03.948863 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:03.948864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:03.948864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:03.948992 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:03.949537 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:03.949545 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:03.949705 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 23:09:03.949807 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 23:09:03.949980 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
I20260809 23:09:03.950800 13357 heartbeater.cc:347] Connected to a master server at 127.8.141.254:38761
I20260809 23:09:03.951063 13357 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:03.951622 13357 heartbeater.cc:511] Master 127.8.141.254:38761 requested a full tablet report, sending...
I20260809 23:09:03.952390 13017 ts_manager.cc:194] Registered new tserver with Master: 8632c8bac103445385ef57fe236248cb (127.8.141.193:35259)
I20260809 23:09:03.953339 13017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:54303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:09:03.978569 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38761
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:04.069172 13364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:04.069476 13364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:04.069525 13364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:04.070799 13364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:04.070883 13364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:04.070930 13364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:04.072547 13364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:04.072865 13364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:04.073129 13364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:04.079336 13370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:04.079521 13364 server_base.cc:1034] running on GCE node
W20260809 23:09:04.079339 13372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:04.079326 13374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:04.080016 13364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:04.080359 13364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:04.081552 13364 hybrid_clock.cc:648] HybridClock initialized: now 1786316944081597 us; error 101 us; skew 500 ppm
Aug 09 23:09:04 dist-test-slave-j2vl krb5kdc[12941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:04.084461 13364 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:09:04.087489 13364 webserver.cc:460] Webserver started at http://127.8.141.194:34447/ using document root <none> and password file <none>
I20260809 23:09:04.087831 13364 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:04.087909 13364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:04.088083 13364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:04.089315 13364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "5d988018c5094c6b9aec16cbc33d2eb9"
format_stamp: "Formatted at 2026-08-09 23:09:04 on dist-test-slave-j2vl"
server_key: "7d2ee15a8a7e2a0ecdc906cb68780a72"
server_key_iv: "7fda2c1d08b44c31fae1222e2423ba57"
server_key_version: "encryptionkey@0"
I20260809 23:09:04.089788 13364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "5d988018c5094c6b9aec16cbc33d2eb9"
format_stamp: "Formatted at 2026-08-09 23:09:04 on dist-test-slave-j2vl"
server_key: "7d2ee15a8a7e2a0ecdc906cb68780a72"
server_key_iv: "7fda2c1d08b44c31fae1222e2423ba57"
server_key_version: "encryptionkey@0"
I20260809 23:09:04.091290 13364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:04.092203 13381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:04.092471 13364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:04.092577 13364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "5d988018c5094c6b9aec16cbc33d2eb9"
format_stamp: "Formatted at 2026-08-09 23:09:04 on dist-test-slave-j2vl"
server_key: "7d2ee15a8a7e2a0ecdc906cb68780a72"
server_key_iv: "7fda2c1d08b44c31fae1222e2423ba57"
server_key_version: "encryptionkey@0"
I20260809 23:09:04.092674 13364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:04.118454 13364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:04.120765 13364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:04.120975 13364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:04.121250 13364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:04.121635 13364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:04.121695 13364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:04.121740 13364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:04.121768 13364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:04.128151 13364 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:36645
I20260809 23:09:04.128196 13560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:36645 every 8 connection(s)
I20260809 23:09:04.128598 13364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:09:04 dist-test-slave-j2vl krb5kdc[12941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:04.135707 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13364
I20260809 23:09:04.135829 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:04.136065 8759 external_mini_cluster.cc:1428] Setting key 5704cb70a0540024e7e32ce142522058
I20260809 23:09:04.139456 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:04.129523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:60143 (local address 127.8.141.254:38761)
0809 23:09:04.129591 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:04.129593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:04.129838 (+ 245us) server_negotiation.cc:407] Connection header received
0809 23:09:04.130378 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:04.130380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:04.130392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:04.130461 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:04.131623 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.132132 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.133360 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.133555 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.133620 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:04.135955 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:04.135956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:04.135957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:04.135965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:04.137816 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:04.138241 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:04.138243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:04.138243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:04.138322 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:04.138659 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:04.138661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:04.138661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:04.138822 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:04.138892 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:04.139289 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 23:09:04.139380 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:09:04.139834 13564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:04.129520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38761 (local address 127.8.141.194:60143)
0809 23:09:04.129681 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:04.129723 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:04.130250 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:04.130549 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:04.130553 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:04.131234 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:04.131494 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:04.131500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.132242 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.132243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:04.133246 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:04.133249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.133644 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.133645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:04.133752 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:04.134278 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:04.134287 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:04.135796 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:04.137917 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:04.137924 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:04.137926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:04.138166 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:04.138417 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:04.138418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:04.138418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:04.138539 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:04.138899 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:04.138907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:04.139101 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 23:09:04.139234 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 23:09:04.139451 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260809 23:09:04.140502 13561 heartbeater.cc:347] Connected to a master server at 127.8.141.254:38761
I20260809 23:09:04.140614 13561 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:04.140849 13561 heartbeater.cc:511] Master 127.8.141.254:38761 requested a full tablet report, sending...
I20260809 23:09:04.141407 13017 ts_manager.cc:194] Registered new tserver with Master: 5d988018c5094c6b9aec16cbc33d2eb9 (127.8.141.194:36645)
I20260809 23:09:04.141985 13017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:60143
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:09:04.172967 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38761
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:04.265973 13570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:04.266216 13570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:04.266243 13570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:04.267498 13570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:04.267546 13570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:04.267575 13570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:04.269246 13570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38761
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:04.269560 13570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:04.269793 13570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:04.275974 13575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:04.276014 13579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:04.276170 13576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:04.276268 13570 server_base.cc:1034] running on GCE node
I20260809 23:09:04.276597 13570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:04.276935 13570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:04.278136 13570 hybrid_clock.cc:648] HybridClock initialized: now 1786316944278044 us; error 120 us; skew 500 ppm
Aug 09 23:09:04 dist-test-slave-j2vl krb5kdc[12941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:04.281163 13570 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:09:04.281792 13570 webserver.cc:460] Webserver started at http://127.8.141.195:41809/ using document root <none> and password file <none>
I20260809 23:09:04.282083 13570 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:04.282162 13570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:04.282325 13570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:04.283517 13570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "3eb57bd80b0642559c4c9937084903ca"
format_stamp: "Formatted at 2026-08-09 23:09:04 on dist-test-slave-j2vl"
server_key: "dab24752948774b98bf16c082e1f4e85"
server_key_iv: "9d7b7008bfeafcef0523a13a002ff254"
server_key_version: "encryptionkey@0"
I20260809 23:09:04.283906 13570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "3eb57bd80b0642559c4c9937084903ca"
format_stamp: "Formatted at 2026-08-09 23:09:04 on dist-test-slave-j2vl"
server_key: "dab24752948774b98bf16c082e1f4e85"
server_key_iv: "9d7b7008bfeafcef0523a13a002ff254"
server_key_version: "encryptionkey@0"
I20260809 23:09:04.285840 13570 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 23:09:04.287508 13588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:04.288023 13570 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 23:09:04.288153 13570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "3eb57bd80b0642559c4c9937084903ca"
format_stamp: "Formatted at 2026-08-09 23:09:04 on dist-test-slave-j2vl"
server_key: "dab24752948774b98bf16c082e1f4e85"
server_key_iv: "9d7b7008bfeafcef0523a13a002ff254"
server_key_version: "encryptionkey@0"
I20260809 23:09:04.288271 13570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:04.328372 13570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:04.331233 13570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:04.331454 13570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:04.331754 13570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:04.332119 13570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:04.332180 13570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:04.332226 13570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:04.332255 13570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:04.339821 13570 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:42231
I20260809 23:09:04.339876 13775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:42231 every 8 connection(s)
I20260809 23:09:04.340287 13570 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:04.340535 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13570
I20260809 23:09:04.340646 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:04.340871 8759 external_mini_cluster.cc:1428] Setting key f0986d78bead5e93a1db4622043564af
Aug 09 23:09:04 dist-test-slave-j2vl krb5kdc[12941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:04.352352 13091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:04.341385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:49507 (local address 127.8.141.254:38761)
0809 23:09:04.341434 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:04.341436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:04.341827 (+ 391us) server_negotiation.cc:407] Connection header received
0809 23:09:04.342446 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:04.342447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:04.342460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:04.342512 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:04.343556 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.344071 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.345129 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.345425 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.345525 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:04.348593 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:04.348595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:04.348596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:04.348606 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:04.350425 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:04.351052 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:04.351054 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:04.351054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:04.351123 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:04.351563 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:04.351564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:04.351565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:04.351721 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:04.351818 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:04.352153 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 23:09:04.352253 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:09:04.352617 13779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:04.341481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38761 (local address 127.8.141.195:49507)
0809 23:09:04.341699 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:04.341740 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:04.342325 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:04.342589 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:04.342593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:04.343210 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:04.343436 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:04.343441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.344175 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.344176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:04.345026 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:04.345029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:04.345542 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:04.345543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:04.345683 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:04.346457 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:04.346473 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:04.348447 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:04.350640 (+ 2193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:04.350654 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:04.350658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:04.350959 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:04.351224 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:04.351226 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:04.351226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:04.351421 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:04.351807 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:04.351816 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:04.351974 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 23:09:04.352100 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 23:09:04.352305 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
I20260809 23:09:04.353574 13777 heartbeater.cc:347] Connected to a master server at 127.8.141.254:38761
I20260809 23:09:04.353737 13777 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:04.353967 13777 heartbeater.cc:511] Master 127.8.141.254:38761 requested a full tablet report, sending...
I20260809 23:09:04.354449 13015 ts_manager.cc:194] Registered new tserver with Master: 3eb57bd80b0642559c4c9937084903ca (127.8.141.195:42231)
I20260809 23:09:04.354883 13015 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:49507
I20260809 23:09:04.364856 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:09:04.381646 13091 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:04.380210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57934 (local address 127.8.141.254:38761)
0809 23:09:04.380310 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:04.380313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:04.380426 (+ 113us) server_negotiation.cc:407] Connection header received
0809 23:09:04.380544 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:04.380545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:04.380561 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:04.380635 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:04.381554 (+ 919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57934: BlockingRecv error: recv got EOF from 127.0.0.1:57934 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 23:09:04.382637 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13129
I20260809 23:09:04.425549 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13364
I20260809 23:09:04.438617 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13570
I20260809 23:09:04.458670 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12963
2026-08-09T23:09:04Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3914 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:04 dist-test-slave-j2vl krb5kdc[13795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:04 dist-test-slave-j2vl krb5kdc[13795](info): set up 2 sockets
Aug 09 23:09:04 dist-test-slave-j2vl krb5kdc[13795](info): commencing operation
krb5kdc: starting...
W20260809 23:09:05.775177 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:05 dist-test-slave-j2vl krb5kdc[13795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:05.917519 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39183
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:44923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:39183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:06.009542 13812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:06.009780 13812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:06.009804 13812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:06.010985 13812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:06.011031 13812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:06.011045 13812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:06.011058 13812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:06.011070 13812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:06.012647 13812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:44923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:39183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:06.012892 13812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:06.013113 13812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:06.019276 13830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:06.019411 13828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:06.019407 13826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:06.019634 13812 server_base.cc:1034] running on GCE node
I20260809 23:09:06.020107 13812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:06.020458 13812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:06.021618 13812 hybrid_clock.cc:648] HybridClock initialized: now 1786316946021589 us; error 42 us; skew 500 ppm
Aug 09 23:09:06 dist-test-slave-j2vl krb5kdc[13795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316946, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:06.024425 13812 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:06.024940 13812 webserver.cc:460] Webserver started at http://127.8.141.254:36113/ using document root <none> and password file <none>
I20260809 23:09:06.025202 13812 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:06.025280 13812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:06.025470 13812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:06.026621 13812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "36c89be984ad40f4ac50162f7513c83b"
format_stamp: "Formatted at 2026-08-09 23:09:06 on dist-test-slave-j2vl"
server_key: "75dd2eb459b8b5bdf6ba245ff14b8fde"
server_key_iv: "37d5e5c4f9840ae2cba54217da893b96"
server_key_version: "encryptionkey@0"
I20260809 23:09:06.027043 13812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "36c89be984ad40f4ac50162f7513c83b"
format_stamp: "Formatted at 2026-08-09 23:09:06 on dist-test-slave-j2vl"
server_key: "75dd2eb459b8b5bdf6ba245ff14b8fde"
server_key_iv: "37d5e5c4f9840ae2cba54217da893b96"
server_key_version: "encryptionkey@0"
I20260809 23:09:06.028744 13812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:06.029639 13839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:06.029829 13812 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:09:06.029953 13812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "36c89be984ad40f4ac50162f7513c83b"
format_stamp: "Formatted at 2026-08-09 23:09:06 on dist-test-slave-j2vl"
server_key: "75dd2eb459b8b5bdf6ba245ff14b8fde"
server_key_iv: "37d5e5c4f9840ae2cba54217da893b96"
server_key_version: "encryptionkey@0"
I20260809 23:09:06.030043 13812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:06.043493 13812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:06.046020 13812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:06.046227 13812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:06.051141 13812 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:39183
I20260809 23:09:06.051177 13913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:39183 every 8 connection(s)
I20260809 23:09:06.051620 13812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:06.052613 13914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:06.054327 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13812
I20260809 23:09:06.054479 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:06.054697 8759 external_mini_cluster.cc:1428] Setting key 5ff7049e73929f97dc900e75db61a5f4
I20260809 23:09:06.055897 13914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b: Bootstrap starting.
I20260809 23:09:06.056699 13914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:06.057053 13914 log.cc:824] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:06.057896 13914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b: No bootstrap required, opened a new log
I20260809 23:09:06.060266 13914 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c89be984ad40f4ac50162f7513c83b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39183 } }
I20260809 23:09:06.060473 13914 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:06.060519 13914 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36c89be984ad40f4ac50162f7513c83b, State: Initialized, Role: FOLLOWER
I20260809 23:09:06.060667 13914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c89be984ad40f4ac50162f7513c83b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39183 } }
I20260809 23:09:06.060783 13914 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:06.060842 13914 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:06.060899 13914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:06.061825 13914 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c89be984ad40f4ac50162f7513c83b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39183 } }
I20260809 23:09:06.061962 13914 leader_election.cc:302] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36c89be984ad40f4ac50162f7513c83b; no voters:
I20260809 23:09:06.062179 13914 leader_election.cc:288] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:06.062263 13923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:06.062476 13914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:06.062505 13923 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [term 1 LEADER]: Becoming Leader. State: Replica: 36c89be984ad40f4ac50162f7513c83b, State: Running, Role: LEADER
I20260809 23:09:06.062747 13923 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c89be984ad40f4ac50162f7513c83b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39183 } }
Aug 09 23:09:06 dist-test-slave-j2vl krb5kdc[13795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:06.063190 13924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36c89be984ad40f4ac50162f7513c83b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c89be984ad40f4ac50162f7513c83b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39183 } } }
I20260809 23:09:06.063275 13924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:06.063715 13926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36c89be984ad40f4ac50162f7513c83b. Latest consensus state: current_term: 1 leader_uuid: "36c89be984ad40f4ac50162f7513c83b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36c89be984ad40f4ac50162f7513c83b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39183 } } }
I20260809 23:09:06.063786 13926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b [sys.catalog]: This master's current role is: LEADER
W20260809 23:09:06.064752 13951 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:09:06.064816 13951 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:09:06.064898 13937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:06.065079 13937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:06.066967 13937 catalog_manager.cc:1347] Generated new cluster ID: b77d471455c2436ea7d83e76d7361d63
I20260809 23:09:06.067032 13937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:06.068112 13918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:06.055303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.8.141.254:39183)
0809 23:09:06.055520 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:06.055530 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:06.055569 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:09:06.056197 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:06.056209 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:06.056283 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:06.056467 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:06.057445 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:06.058607 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:06.059840 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:06.060307 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:06.060409 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:06.063955 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:06.063960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:06.063960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:06.063976 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:06.066283 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:06.066746 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:06.066750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:06.066750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:06.066854 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:06.067112 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:06.067113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:06.067113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:06.067425 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:06.067567 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:06.067747 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 23:09:06.067889 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260809 23:09:06.069139 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:09:06.086637 13937 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:06.087378 13937 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:06.088770 13918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:06.084394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.8.141.254:39183)
0809 23:09:06.084445 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:06.084448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:06.084575 (+ 127us) server_negotiation.cc:407] Connection header received
0809 23:09:06.084645 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:06.084645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:06.084657 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:06.084735 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:06.085229 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:06.085982 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:06.087007 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:06.087319 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:06.087361 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:06.088242 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:06.088243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:06.088244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:06.088475 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:06.088494 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:06.088501 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:06.088531 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:06.088635 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 23:09:06.088685 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:09:06.093133 13937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36c89be984ad40f4ac50162f7513c83b: Generated new TSK 0
I20260809 23:09:06.093428 13937 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:09:06.101686 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13812
2026-08-09T23:09:06Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1640 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:06 dist-test-slave-j2vl krb5kdc[13969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:06 dist-test-slave-j2vl krb5kdc[13969](info): set up 2 sockets
Aug 09 23:09:06 dist-test-slave-j2vl krb5kdc[13969](info): commencing operation
krb5kdc: starting...
W20260809 23:09:07.427995 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:07 dist-test-slave-j2vl krb5kdc[13969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:07.573590 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36943
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:36943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:07.670411 13995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:07.670651 13995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:07.670678 13995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:07.671897 13995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:07.671947 13995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:07.671962 13995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:07.671975 13995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:07.671988 13995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:07.673638 13995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:36943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:07.673895 13995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:07.674103 13995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:07.680188 14007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:07.680212 14012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:07.680188 14009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:07.680328 13995 server_base.cc:1034] running on GCE node
I20260809 23:09:07.681046 13995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:07.681571 13995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:07.682883 13995 hybrid_clock.cc:648] HybridClock initialized: now 1786316947682847 us; error 49 us; skew 500 ppm
Aug 09 23:09:07 dist-test-slave-j2vl krb5kdc[13969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316947, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:07.685961 13995 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:07.686431 13995 webserver.cc:460] Webserver started at http://127.8.141.254:34071/ using document root <none> and password file <none>
I20260809 23:09:07.686798 13995 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:07.686865 13995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:07.687048 13995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:07.688220 13995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "aef1a07156a34177bf0ab79a45e03a15"
format_stamp: "Formatted at 2026-08-09 23:09:07 on dist-test-slave-j2vl"
server_key: "2b361175c4fcecbb3b30b4fde8299737"
server_key_iv: "5b274ee72fae5a8dba190a7b34fa2435"
server_key_version: "encryptionkey@0"
I20260809 23:09:07.688612 13995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "aef1a07156a34177bf0ab79a45e03a15"
format_stamp: "Formatted at 2026-08-09 23:09:07 on dist-test-slave-j2vl"
server_key: "2b361175c4fcecbb3b30b4fde8299737"
server_key_iv: "5b274ee72fae5a8dba190a7b34fa2435"
server_key_version: "encryptionkey@0"
I20260809 23:09:07.690197 13995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:07.691092 14023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:07.691295 13995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:07.691414 13995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "aef1a07156a34177bf0ab79a45e03a15"
format_stamp: "Formatted at 2026-08-09 23:09:07 on dist-test-slave-j2vl"
server_key: "2b361175c4fcecbb3b30b4fde8299737"
server_key_iv: "5b274ee72fae5a8dba190a7b34fa2435"
server_key_version: "encryptionkey@0"
I20260809 23:09:07.691591 13995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:07.702993 13995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:07.705730 13995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:07.705925 13995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:07.710242 13995 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:36943
I20260809 23:09:07.710333 14123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:36943 every 8 connection(s)
I20260809 23:09:07.710785 13995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:07.711731 14124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:07.714658 14124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15: Bootstrap starting.
I20260809 23:09:07.715374 14124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:07.715723 14124 log.cc:824] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:07.716599 14124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15: No bootstrap required, opened a new log
I20260809 23:09:07.718418 14124 raft_consensus.cc:348] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef1a07156a34177bf0ab79a45e03a15" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36943 } }
I20260809 23:09:07.718601 14124 raft_consensus.cc:374] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:07.718648 14124 raft_consensus.cc:729] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aef1a07156a34177bf0ab79a45e03a15, State: Initialized, Role: FOLLOWER
I20260809 23:09:07.718861 14124 consensus_queue.cc:260] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef1a07156a34177bf0ab79a45e03a15" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36943 } }
I20260809 23:09:07.718974 14124 raft_consensus.cc:388] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:07.719018 14124 raft_consensus.cc:482] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:07.719074 14124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:07.720052 14124 raft_consensus.cc:504] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef1a07156a34177bf0ab79a45e03a15" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36943 } }
I20260809 23:09:07.720187 14124 leader_election.cc:302] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aef1a07156a34177bf0ab79a45e03a15; no voters:
I20260809 23:09:07.720392 14124 leader_election.cc:288] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:07.720579 14132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:07.720741 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13995
I20260809 23:09:07.720804 14132 raft_consensus.cc:686] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [term 1 LEADER]: Becoming Leader. State: Replica: aef1a07156a34177bf0ab79a45e03a15, State: Running, Role: LEADER
I20260809 23:09:07.720851 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:07.720836 14124 sys_catalog.cc:564] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:07.720917 14132 consensus_queue.cc:237] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef1a07156a34177bf0ab79a45e03a15" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36943 } }
I20260809 23:09:07.721165 8759 external_mini_cluster.cc:1428] Setting key 011c3b5feed6c691111a9ed7c203bd1d
I20260809 23:09:07.721279 14132 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aef1a07156a34177bf0ab79a45e03a15. Latest consensus state: current_term: 1 leader_uuid: "aef1a07156a34177bf0ab79a45e03a15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef1a07156a34177bf0ab79a45e03a15" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36943 } } }
I20260809 23:09:07.721369 14132 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:07.721504 14133 sys_catalog.cc:455] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aef1a07156a34177bf0ab79a45e03a15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aef1a07156a34177bf0ab79a45e03a15" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36943 } } }
I20260809 23:09:07.721565 14133 sys_catalog.cc:458] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:07.722265 14141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:07.723210 14141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:07.725838 14141 catalog_manager.cc:1347] Generated new cluster ID: dfc4a4954ea84af8bc6221c3d0a03f0e
I20260809 23:09:07.725912 14141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:09:07 dist-test-slave-j2vl krb5kdc[13969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:07.733805 14141 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:07.734253 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:07.721739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.8.141.254:36943)
0809 23:09:07.723068 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:07.723079 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:07.723120 (+ 41us) server_negotiation.cc:407] Connection header received
0809 23:09:07.723258 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:07.723266 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:07.723336 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:07.723503 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:07.724318 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:07.725440 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:07.726399 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:07.726895 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:07.727037 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:07.729620 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:07.729626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:07.729626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:07.729642 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:07.731544 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:07.732129 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:07.732133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:07.732134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:07.732271 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:07.732541 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:07.732543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:07.732543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:07.733077 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:07.733408 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:07.733585 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 23:09:07.733841 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":85,"threads_started":1}
I20260809 23:09:07.734560 14141 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:07.746744 14141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aef1a07156a34177bf0ab79a45e03a15: Generated new TSK 0
I20260809 23:09:07.747006 14141 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:09:07.782290 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36943
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:07.877177 14167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:07.877472 14167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:07.877506 14167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:07.879081 14167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:07.879148 14167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:07.879184 14167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:07.880999 14167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:07.881414 14167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:07.881758 14167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:07.888233 14174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:07.888257 14167 server_base.cc:1034] running on GCE node
W20260809 23:09:07.888226 14176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:07.888233 14178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:07.888797 14167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:07.889174 14167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:07.890375 14167 hybrid_clock.cc:648] HybridClock initialized: now 1786316947890344 us; error 43 us; skew 500 ppm
Aug 09 23:09:07 dist-test-slave-j2vl krb5kdc[13969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316947, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:07.893787 14167 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:09:07.894790 14167 webserver.cc:460] Webserver started at http://127.8.141.193:46105/ using document root <none> and password file <none>
I20260809 23:09:07.895210 14167 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:07.895448 14167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:07.895906 14167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:07.897151 14167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "882f120d1f4c47f49be97ff995a09d7d"
format_stamp: "Formatted at 2026-08-09 23:09:07 on dist-test-slave-j2vl"
server_key: "c8faca15522a3495783d03a01d24cc05"
server_key_iv: "bb3f4d50b382aab47e2e2db19dd6dd7d"
server_key_version: "encryptionkey@0"
I20260809 23:09:07.897748 14167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "882f120d1f4c47f49be97ff995a09d7d"
format_stamp: "Formatted at 2026-08-09 23:09:07 on dist-test-slave-j2vl"
server_key: "c8faca15522a3495783d03a01d24cc05"
server_key_iv: "bb3f4d50b382aab47e2e2db19dd6dd7d"
server_key_version: "encryptionkey@0"
I20260809 23:09:07.899537 14167 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 23:09:07.900729 14194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:07.900991 14167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:07.901111 14167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "882f120d1f4c47f49be97ff995a09d7d"
format_stamp: "Formatted at 2026-08-09 23:09:07 on dist-test-slave-j2vl"
server_key: "c8faca15522a3495783d03a01d24cc05"
server_key_iv: "bb3f4d50b382aab47e2e2db19dd6dd7d"
server_key_version: "encryptionkey@0"
I20260809 23:09:07.901226 14167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:07.920111 14167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:07.922963 14167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:07.923161 14167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:07.923470 14167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:07.923852 14167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:07.923916 14167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:07.923957 14167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:07.923993 14167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:07.930902 14167 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:37901
I20260809 23:09:07.930982 14378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:37901 every 8 connection(s)
I20260809 23:09:07.931360 14167 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:09:07 dist-test-slave-j2vl krb5kdc[13969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316947, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:07.939600 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 14167
I20260809 23:09:07.939735 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:07.939976 8759 external_mini_cluster.cc:1428] Setting key e2d0e03f78001ebf5217298a370ee62f
I20260809 23:09:07.942282 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:07.932275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:50005 (local address 127.8.141.254:36943)
0809 23:09:07.932330 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:07.932333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:07.932597 (+ 264us) server_negotiation.cc:407] Connection header received
0809 23:09:07.933138 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:07.933139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:07.933154 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:07.933222 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:07.934356 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:07.934944 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:07.936074 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:07.936298 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:07.936360 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:07.938890 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:07.938892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:07.938892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:07.938900 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:07.940589 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:07.941126 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:07.941128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:07.941128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:07.941200 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:07.941492 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:07.941492 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:07.941493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:07.941652 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:07.941737 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:07.942033 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 23:09:07.942140 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:09:07.942355 14381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:07.932224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36943 (local address 127.8.141.193:50005)
0809 23:09:07.932424 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:07.932483 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:07.933037 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:07.933279 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:07.933283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:07.934018 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:07.934244 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:07.934249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:07.935069 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:07.935070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:07.935976 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:07.935979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:07.936386 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:07.936387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:07.936500 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:07.937148 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:07.937157 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:07.938786 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:07.940745 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:07.940754 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:07.940756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:07.941044 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:07.941280 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:07.941281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:07.941281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:07.941425 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:07.941722 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:07.941728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:07.941863 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 23:09:07.941969 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 23:09:07.942124 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
I20260809 23:09:07.943123 14379 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36943
I20260809 23:09:07.943231 14379 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:07.943495 14379 heartbeater.cc:511] Master 127.8.141.254:36943 requested a full tablet report, sending...
I20260809 23:09:07.944195 14054 ts_manager.cc:194] Registered new tserver with Master: 882f120d1f4c47f49be97ff995a09d7d (127.8.141.193:37901)
I20260809 23:09:07.945250 14054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:50005
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:09:07.974126 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36943
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:08.071614 14386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:08.071851 14386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:08.071878 14386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:08.073175 14386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:08.073236 14386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:08.073266 14386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:08.074990 14386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:08.075292 14386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:08.075574 14386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:08.081877 14402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:08.081871 14396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:08.081911 14386 server_base.cc:1034] running on GCE node
W20260809 23:09:08.081871 14398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:08.082403 14386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:08.082716 14386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:08.083897 14386 hybrid_clock.cc:648] HybridClock initialized: now 1786316948083867 us; error 48 us; skew 500 ppm
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[13969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316948, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:08.087154 14386 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:09:08.087625 14386 webserver.cc:460] Webserver started at http://127.8.141.194:45085/ using document root <none> and password file <none>
I20260809 23:09:08.087841 14386 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:08.087886 14386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:08.087987 14386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:08.089071 14386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "2484c1c2d67743b0be05cfeeb3aa4cd2"
format_stamp: "Formatted at 2026-08-09 23:09:08 on dist-test-slave-j2vl"
server_key: "3c3baad8b199d537dde5413468456776"
server_key_iv: "fe5db97e6ceedb89d9b69ed3968713d9"
server_key_version: "encryptionkey@0"
I20260809 23:09:08.089495 14386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "2484c1c2d67743b0be05cfeeb3aa4cd2"
format_stamp: "Formatted at 2026-08-09 23:09:08 on dist-test-slave-j2vl"
server_key: "3c3baad8b199d537dde5413468456776"
server_key_iv: "fe5db97e6ceedb89d9b69ed3968713d9"
server_key_version: "encryptionkey@0"
I20260809 23:09:08.091110 14386 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:08.092074 14416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:08.092348 14386 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:08.092450 14386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "2484c1c2d67743b0be05cfeeb3aa4cd2"
format_stamp: "Formatted at 2026-08-09 23:09:08 on dist-test-slave-j2vl"
server_key: "3c3baad8b199d537dde5413468456776"
server_key_iv: "fe5db97e6ceedb89d9b69ed3968713d9"
server_key_version: "encryptionkey@0"
I20260809 23:09:08.092610 14386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:08.117086 14386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:08.119707 14386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:08.119853 14386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:08.120069 14386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:08.120460 14386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:08.120507 14386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:08.120532 14386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:08.120544 14386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:08.126909 14386 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:38449
I20260809 23:09:08.127023 14600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:38449 every 8 connection(s)
I20260809 23:09:08.127357 14386 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:08.131932 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 14386
I20260809 23:09:08.132104 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:08.132311 8759 external_mini_cluster.cc:1428] Setting key 161180f29bb3ff1df7cf6b1e426f4d5c
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[13969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316948, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:08.139726 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:08.128223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:56865 (local address 127.8.141.254:36943)
0809 23:09:08.128300 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:08.128303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:08.128584 (+ 281us) server_negotiation.cc:407] Connection header received
0809 23:09:08.129277 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:08.129278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:08.129293 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:08.129431 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:08.130665 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.131256 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.132733 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.132992 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.133078 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:08.135996 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:08.135999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:08.135999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:08.136008 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:08.137928 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:08.138436 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:08.138437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:08.138438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:08.138507 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:08.138877 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:08.138878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:08.138878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:08.139046 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:08.139128 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:08.139468 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 23:09:08.139583 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:09:08.139756 14605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:08.128223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36943 (local address 127.8.141.194:56865)
0809 23:09:08.128439 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:08.128479 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:08.129143 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:08.129512 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:08.129517 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:08.130351 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:08.130562 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:08.130567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.131413 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.131414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:08.132638 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:08.132642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.133102 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.133103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:08.133274 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:08.134095 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:08.134106 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:08.135861 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:08.138055 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:08.138065 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:08.138068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:08.138344 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:08.138622 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:08.138624 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:08.138624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:08.138785 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:08.139144 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:08.139152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:08.139293 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 23:09:08.139411 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 23:09:08.139570 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":120,"threads_started":1}
I20260809 23:09:08.140427 14601 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36943
I20260809 23:09:08.140537 14601 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:08.140761 14601 heartbeater.cc:511] Master 127.8.141.254:36943 requested a full tablet report, sending...
I20260809 23:09:08.141248 14054 ts_manager.cc:194] Registered new tserver with Master: 2484c1c2d67743b0be05cfeeb3aa4cd2 (127.8.141.194:38449)
I20260809 23:09:08.141669 14054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:56865
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:09:08.169099 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36943
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:08.260150 14612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:08.260418 14612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:08.260461 14612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:08.261767 14612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:08.261838 14612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:08.261886 14612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:08.263448 14612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:08.263737 14612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:08.264007 14612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:08.270756 14622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:08.270756 14621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:08.270828 14624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:08.270849 14612 server_base.cc:1034] running on GCE node
I20260809 23:09:08.271212 14612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:08.271541 14612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:08.272725 14612 hybrid_clock.cc:648] HybridClock initialized: now 1786316948272703 us; error 32 us; skew 500 ppm
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[13969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316948, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:08.275589 14612 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:09:08.276176 14612 webserver.cc:460] Webserver started at http://127.8.141.195:43507/ using document root <none> and password file <none>
I20260809 23:09:08.276451 14612 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:08.276503 14612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:08.276687 14612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:08.277906 14612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "cd4274a7c38b4490a8301145c66047c5"
format_stamp: "Formatted at 2026-08-09 23:09:08 on dist-test-slave-j2vl"
server_key: "7778cd5bbfaa71e37370787064827d2b"
server_key_iv: "59ad02c3e3b5d2210260f497ef8c8169"
server_key_version: "encryptionkey@0"
I20260809 23:09:08.278330 14612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "cd4274a7c38b4490a8301145c66047c5"
format_stamp: "Formatted at 2026-08-09 23:09:08 on dist-test-slave-j2vl"
server_key: "7778cd5bbfaa71e37370787064827d2b"
server_key_iv: "59ad02c3e3b5d2210260f497ef8c8169"
server_key_version: "encryptionkey@0"
I20260809 23:09:08.279899 14612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:08.280755 14636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:08.280984 14612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:08.281105 14612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "cd4274a7c38b4490a8301145c66047c5"
format_stamp: "Formatted at 2026-08-09 23:09:08 on dist-test-slave-j2vl"
server_key: "7778cd5bbfaa71e37370787064827d2b"
server_key_iv: "59ad02c3e3b5d2210260f497ef8c8169"
server_key_version: "encryptionkey@0"
I20260809 23:09:08.281194 14612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:08.305446 14612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:08.307994 14612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:08.308194 14612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:08.308490 14612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:08.308846 14612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:08.308904 14612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:08.308946 14612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:08.308977 14612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:08.316269 14612 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:34723
I20260809 23:09:08.316330 14818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:34723 every 8 connection(s)
I20260809 23:09:08.316751 14612 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[13969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316948, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:08.326315 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 14612
I20260809 23:09:08.326431 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:08.326643 8759 external_mini_cluster.cc:1428] Setting key 5d52e77195805bc9595a525a4ea85701
I20260809 23:09:08.327108 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:08.317707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:47791 (local address 127.8.141.254:36943)
0809 23:09:08.317791 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:08.317795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:08.318097 (+ 302us) server_negotiation.cc:407] Connection header received
0809 23:09:08.318624 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:08.318626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:08.318638 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:08.318713 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:08.319804 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.320333 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.321544 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.321741 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.321809 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:08.324235 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:08.324236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:08.324236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:08.324244 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:08.325579 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:08.325978 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:08.325979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:08.325979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:08.326030 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:08.326330 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:08.326331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:08.326331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:08.326497 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:08.326553 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:08.326902 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 23:09:08.326993 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:09:08.327334 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:08.317695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36943 (local address 127.8.141.195:47791)
0809 23:09:08.317947 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:08.317989 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:08.318505 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:08.318771 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:08.318774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:08.319465 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:08.319672 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:08.319677 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.320462 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.320463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:08.321406 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:08.321410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.321826 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.321827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:08.321950 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:08.322599 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:08.322609 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:08.324105 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:08.325660 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:08.325667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:08.325669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:08.325913 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:08.326088 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:08.326089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:08.326089 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:08.326260 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:08.326551 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:08.326559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:08.326737 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 23:09:08.326852 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 23:09:08.327068 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":126,"threads_started":1}
I20260809 23:09:08.327917 14820 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36943
I20260809 23:09:08.328006 14820 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:08.328164 14820 heartbeater.cc:511] Master 127.8.141.254:36943 requested a full tablet report, sending...
I20260809 23:09:08.328497 14054 ts_manager.cc:194] Registered new tserver with Master: cd4274a7c38b4490a8301145c66047c5 (127.8.141.195:34723)
I20260809 23:09:08.328882 14054 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:47791
I20260809 23:09:08.338745 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[13969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316948, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[13969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316948, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:08.364230 14144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:08.356511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.8.141.254:36943)
0809 23:09:08.356612 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:08.356615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:08.356792 (+ 177us) server_negotiation.cc:407] Connection header received
0809 23:09:08.356896 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:08.356897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:08.356909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:08.357006 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:08.357720 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.358230 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.359024 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:08.359224 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:08.359304 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:08.361724 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:08.361726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:08.361726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:08.361734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:08.363058 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:08.363435 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:08.363436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:08.363436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:08.363508 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:08.363681 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:08.363682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:08.363682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:08.363835 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:08.363914 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:08.364064 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 23:09:08.364127 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 23:09:08.364535 14054 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54560
I20260809 23:09:08.365024 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 14167
I20260809 23:09:08.381247 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 14386
I20260809 23:09:08.392570 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 14612
I20260809 23:09:08.406512 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13995
2026-08-09T23:09:08Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2324 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[14852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[14852](info): set up 2 sockets
Aug 09 23:09:08 dist-test-slave-j2vl krb5kdc[14852](info): commencing operation
krb5kdc: starting...
W20260809 23:09:09.745208 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:09 dist-test-slave-j2vl krb5kdc[14852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:09.884919 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41499
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:41499
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:09.972606 14876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:09.972849 14876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:09.972874 14876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:09.974104 14876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:09.974154 14876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:09.974169 14876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:09.974182 14876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:09.974195 14876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:09.975772 14876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:41499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:09.976020 14876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:09.976218 14876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:09.981696 14886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:09.981777 14888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:09.981775 14885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:09.982547 14876 server_base.cc:1034] running on GCE node
I20260809 23:09:09.982901 14876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:09.983264 14876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:09.984549 14876 hybrid_clock.cc:648] HybridClock initialized: now 1786316949984516 us; error 51 us; skew 500 ppm
Aug 09 23:09:09 dist-test-slave-j2vl krb5kdc[14852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316949, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:09.993228 14876 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:09.993921 14876 webserver.cc:460] Webserver started at http://127.8.141.254:38797/ using document root <none> and password file <none>
I20260809 23:09:09.994194 14876 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:09.994273 14876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:09.994445 14876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:09.995755 14876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "bb7abce399f64911a2b43e16d22956ac"
format_stamp: "Formatted at 2026-08-09 23:09:09 on dist-test-slave-j2vl"
server_key: "9d9f3738d73f8f05a63c4c716babe5cd"
server_key_iv: "2003dd9e69a25567211ec9123c60f633"
server_key_version: "encryptionkey@0"
I20260809 23:09:09.996419 14876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "bb7abce399f64911a2b43e16d22956ac"
format_stamp: "Formatted at 2026-08-09 23:09:09 on dist-test-slave-j2vl"
server_key: "9d9f3738d73f8f05a63c4c716babe5cd"
server_key_iv: "2003dd9e69a25567211ec9123c60f633"
server_key_version: "encryptionkey@0"
I20260809 23:09:09.998196 14876 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260809 23:09:09.999154 14902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:09.999449 14876 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:09.999605 14876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "bb7abce399f64911a2b43e16d22956ac"
format_stamp: "Formatted at 2026-08-09 23:09:09 on dist-test-slave-j2vl"
server_key: "9d9f3738d73f8f05a63c4c716babe5cd"
server_key_iv: "2003dd9e69a25567211ec9123c60f633"
server_key_version: "encryptionkey@0"
I20260809 23:09:09.999697 14876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:10.008653 14876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:10.011309 14876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:10.011504 14876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:10.016275 14876 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:41499
I20260809 23:09:10.016337 14995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:41499 every 8 connection(s)
I20260809 23:09:10.016913 14876 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:10.017743 15000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:10.020229 15000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac: Bootstrap starting.
I20260809 23:09:10.020937 15000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:10.021313 15000 log.cc:824] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:10.021490 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 14876
I20260809 23:09:10.021624 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:10.021891 8759 external_mini_cluster.cc:1428] Setting key b7b51d12fd15a52f8c16665b4181cfe7
I20260809 23:09:10.022465 15000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac: No bootstrap required, opened a new log
I20260809 23:09:10.025498 15000 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7abce399f64911a2b43e16d22956ac" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41499 } }
I20260809 23:09:10.025709 15000 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:10.025748 15000 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb7abce399f64911a2b43e16d22956ac, State: Initialized, Role: FOLLOWER
I20260809 23:09:10.025877 15000 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7abce399f64911a2b43e16d22956ac" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41499 } }
I20260809 23:09:10.025983 15000 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:10.026018 15000 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:10.026057 15000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:10.026894 15000 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7abce399f64911a2b43e16d22956ac" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41499 } }
I20260809 23:09:10.027009 15000 leader_election.cc:302] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb7abce399f64911a2b43e16d22956ac; no voters:
I20260809 23:09:10.027241 15000 leader_election.cc:288] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:10.027321 15008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:10.027509 15008 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [term 1 LEADER]: Becoming Leader. State: Replica: bb7abce399f64911a2b43e16d22956ac, State: Running, Role: LEADER
I20260809 23:09:10.027563 15000 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:10.027724 15008 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7abce399f64911a2b43e16d22956ac" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41499 } }
I20260809 23:09:10.028494 15013 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb7abce399f64911a2b43e16d22956ac. Latest consensus state: current_term: 1 leader_uuid: "bb7abce399f64911a2b43e16d22956ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7abce399f64911a2b43e16d22956ac" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41499 } } }
I20260809 23:09:10.028605 15013 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:10.028743 15012 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb7abce399f64911a2b43e16d22956ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb7abce399f64911a2b43e16d22956ac" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41499 } } }
I20260809 23:09:10.028944 15012 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac [sys.catalog]: This master's current role is: LEADER
W20260809 23:09:10.029914 15038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:09:10.029980 15038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:09:10.030035 15026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:10.030227 15026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:10.031950 15026 catalog_manager.cc:1347] Generated new cluster ID: 52e9295a4ba34c94bf74411a4c6aa0fd
I20260809 23:09:10.032006 15026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:10.036242 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.022577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.8.141.254:41499)
0809 23:09:10.022810 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:10.022820 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:10.022856 (+ 36us) server_negotiation.cc:407] Connection header received
0809 23:09:10.023995 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:10.024007 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:10.024080 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:10.024259 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:10.025058 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.026515 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.027601 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.028589 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.028724 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.032048 (+ 3324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:10.032052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:10.032052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:10.032066 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:10.034193 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.034690 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.034693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.034694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.034824 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.035072 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.035072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.035073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.035407 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:10.035555 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:10.035715 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 23:09:10.036000 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":107,"threads_started":1}
I20260809 23:09:10.038689 15026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:10.039481 15026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:10.042855 15026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb7abce399f64911a2b43e16d22956ac: Generated new TSK 0
I20260809 23:09:10.043051 15026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:09:10.077396 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41499
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:10.165838 15044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:10.166108 15044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:10.166153 15044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:10.167430 15044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:10.167513 15044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:10.167564 15044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:10.169248 15044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:10.169561 15044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:10.169828 15044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:10.175334 15050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:10.175355 15049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:10.175334 15054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:10.175725 15044 server_base.cc:1034] running on GCE node
I20260809 23:09:10.175989 15044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:10.176318 15044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:10.177445 15044 hybrid_clock.cc:648] HybridClock initialized: now 1786316950177446 us; error 82 us; skew 500 ppm
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:10.180137 15044 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:09:10.180702 15044 webserver.cc:460] Webserver started at http://127.8.141.193:33699/ using document root <none> and password file <none>
I20260809 23:09:10.180975 15044 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:10.181051 15044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:10.181211 15044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:10.182415 15044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "5b50c1b53f054118b8097d78c4c8e2e6"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "7edf3d635498b852df5e30e054de94b6"
server_key_iv: "cd81eab94898302f04cc73b33f0d5d17"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.182808 15044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "5b50c1b53f054118b8097d78c4c8e2e6"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "7edf3d635498b852df5e30e054de94b6"
server_key_iv: "cd81eab94898302f04cc73b33f0d5d17"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.184648 15044 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 23:09:10.185639 15065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.185876 15044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:10.186002 15044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "5b50c1b53f054118b8097d78c4c8e2e6"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "7edf3d635498b852df5e30e054de94b6"
server_key_iv: "cd81eab94898302f04cc73b33f0d5d17"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.186089 15044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:10.198716 15044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:10.200696 15044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:10.200877 15044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:10.201141 15044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:10.201586 15044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:10.201646 15044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.201690 15044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:10.201719 15044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.208717 15044 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:34749
I20260809 23:09:10.208760 15263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:34749 every 8 connection(s)
I20260809 23:09:10.209158 15044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:10.214232 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 15044
I20260809 23:09:10.214383 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:10.214946 8759 external_mini_cluster.cc:1428] Setting key 54f517497eb29278f5741aca7ef4be9c
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:10.220279 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.210049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:47155 (local address 127.8.141.254:41499)
0809 23:09:10.210115 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:10.210118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:10.210330 (+ 212us) server_negotiation.cc:407] Connection header received
0809 23:09:10.211253 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:10.211255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:10.211267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:10.211334 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:10.212039 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.212604 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.213694 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.213913 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.213999 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.216799 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:10.216801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:10.216801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:10.216809 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:10.218628 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.219145 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.219147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.219147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.219217 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.219506 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.219508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.219508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.219645 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:10.219733 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:10.220108 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 23:09:10.220206 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:09:10.220561 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.209982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41499 (local address 127.8.141.193:47155)
0809 23:09:10.210158 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:10.210204 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:10.211144 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:10.211421 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:10.211425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:10.211699 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:10.211922 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:10.211926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.212693 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.212693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:10.213612 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:10.213615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.214055 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.214057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:10.214227 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.214905 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:10.214917 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:10.216683 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:10.218758 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:10.218769 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:10.218772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:10.219055 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:10.219296 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:10.219297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:10.219297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:10.219424 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:10.219721 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:10.219730 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:10.219919 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 23:09:10.220044 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 23:09:10.220244 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260809 23:09:10.221194 15264 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41499
I20260809 23:09:10.221304 15264 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:10.221560 15264 heartbeater.cc:511] Master 127.8.141.254:41499 requested a full tablet report, sending...
I20260809 23:09:10.222199 14931 ts_manager.cc:194] Registered new tserver with Master: 5b50c1b53f054118b8097d78c4c8e2e6 (127.8.141.193:34749)
I20260809 23:09:10.222833 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:47155
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:09:10.250254 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41499
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:10.343405 15270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:10.343688 15270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:10.343732 15270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:10.344995 15270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:10.345070 15270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:10.345117 15270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:10.346910 15270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:10.347209 15270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:10.347465 15270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:10.353292 15279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:10.353294 15276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:10.353634 15270 server_base.cc:1034] running on GCE node
W20260809 23:09:10.353292 15275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:10.353932 15270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:10.354234 15270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:10.355424 15270 hybrid_clock.cc:648] HybridClock initialized: now 1786316950355428 us; error 50 us; skew 500 ppm
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:10.358520 15270 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:09:10.359002 15270 webserver.cc:460] Webserver started at http://127.8.141.194:37469/ using document root <none> and password file <none>
I20260809 23:09:10.359272 15270 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:10.359333 15270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:10.359473 15270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:10.360615 15270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "c887d3c07cc04fd48af3f3256e877f63"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "03a0829cb470e8c09a78200fbb7f37c4"
server_key_iv: "8dfcb08feaaa6dc7b8e7e5ff323c007e"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.361009 15270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "c887d3c07cc04fd48af3f3256e877f63"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "03a0829cb470e8c09a78200fbb7f37c4"
server_key_iv: "8dfcb08feaaa6dc7b8e7e5ff323c007e"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.362660 15270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:10.363611 15293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.363905 15270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:10.363999 15270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "c887d3c07cc04fd48af3f3256e877f63"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "03a0829cb470e8c09a78200fbb7f37c4"
server_key_iv: "8dfcb08feaaa6dc7b8e7e5ff323c007e"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.364108 15270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:10.372588 15270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:10.375617 15270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:10.375816 15270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:10.376065 15270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:10.376423 15270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:10.376466 15270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.376511 15270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:10.376536 15270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.382791 15270 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:35415
I20260809 23:09:10.382829 15490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:35415 every 8 connection(s)
I20260809 23:09:10.383401 15270 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:10.387306 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 15270
I20260809 23:09:10.387446 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:10.387640 8759 external_mini_cluster.cc:1428] Setting key 298aa8b69e5ac2eab0520a2591551dee
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:10.394897 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.384337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:40447 (local address 127.8.141.254:41499)
0809 23:09:10.384418 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:10.384421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:10.384689 (+ 268us) server_negotiation.cc:407] Connection header received
0809 23:09:10.385789 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:10.385791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:10.385803 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:10.385876 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:10.386645 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.387154 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.388713 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.388935 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.389056 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.391776 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:10.391778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:10.391779 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:10.391794 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:10.393281 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.393818 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.393820 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.393820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.393883 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.394155 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.394157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.394157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.394285 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:10.394380 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:10.394687 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 23:09:10.394791 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:09:10.395102 15495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.384318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41499 (local address 127.8.141.194:40447)
0809 23:09:10.384553 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:10.384600 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:10.385643 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:10.386000 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:10.386004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:10.386299 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:10.386522 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:10.386527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.387275 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.387277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:10.388570 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:10.388574 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.389071 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.389072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:10.389226 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.389898 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:10.389907 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:10.391662 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:10.393461 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:10.393470 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:10.393473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:10.393727 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:10.393953 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:10.393953 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:10.393954 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:10.394078 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:10.394370 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:10.394378 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:10.394536 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 23:09:10.394650 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:09:10.394844 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
I20260809 23:09:10.395763 15492 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41499
I20260809 23:09:10.395890 15492 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:10.396111 15492 heartbeater.cc:511] Master 127.8.141.254:41499 requested a full tablet report, sending...
I20260809 23:09:10.396586 14931 ts_manager.cc:194] Registered new tserver with Master: c887d3c07cc04fd48af3f3256e877f63 (127.8.141.194:35415)
I20260809 23:09:10.397002 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:40447
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:09:10.421586 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41499
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:10.518167 15500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:10.518435 15500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:10.518466 15500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:10.519954 15500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:10.520012 15500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:10.520046 15500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:10.522038 15500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41499
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:10.522385 15500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:10.522687 15500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:10.528868 15511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:10.528918 15500 server_base.cc:1034] running on GCE node
W20260809 23:09:10.528926 15514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:10.528880 15510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:10.529433 15500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:10.529757 15500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:10.530961 15500 hybrid_clock.cc:648] HybridClock initialized: now 1786316950530942 us; error 43 us; skew 500 ppm
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:10.533825 15500 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:09:10.534389 15500 webserver.cc:460] Webserver started at http://127.8.141.195:33551/ using document root <none> and password file <none>
I20260809 23:09:10.534638 15500 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:10.534686 15500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:10.534811 15500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:10.535907 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "70b56a73e6ba464d99280ddc1734c021"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "88ad8f30252e0e398ca4b5469e1c66d1"
server_key_iv: "e921ac8795265e5844f97fc731b5ec8d"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.536370 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "70b56a73e6ba464d99280ddc1734c021"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "88ad8f30252e0e398ca4b5469e1c66d1"
server_key_iv: "e921ac8795265e5844f97fc731b5ec8d"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.538033 15500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:10.538944 15529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.539165 15500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:10.539285 15500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "70b56a73e6ba464d99280ddc1734c021"
format_stamp: "Formatted at 2026-08-09 23:09:10 on dist-test-slave-j2vl"
server_key: "88ad8f30252e0e398ca4b5469e1c66d1"
server_key_iv: "e921ac8795265e5844f97fc731b5ec8d"
server_key_version: "encryptionkey@0"
I20260809 23:09:10.539373 15500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:10.551559 15500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:10.554046 15500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:10.554232 15500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:10.554509 15500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:10.554867 15500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:10.554927 15500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.554975 15500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:10.555009 15500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:10.562345 15500 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:44119
I20260809 23:09:10.562386 15720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:44119 every 8 connection(s)
I20260809 23:09:10.562776 15500 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:10.568655 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 15500
I20260809 23:09:10.568750 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:10.569000 8759 external_mini_cluster.cc:1428] Setting key a287a51a0f042413a68e9f6cb4364cfb
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:10.573585 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.563589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:41451 (local address 127.8.141.254:41499)
0809 23:09:10.563654 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:10.563656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:10.563896 (+ 240us) server_negotiation.cc:407] Connection header received
0809 23:09:10.564958 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:10.564960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:10.564975 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:10.565040 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:10.565796 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.566316 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.567362 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.567560 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.567644 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.570533 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:10.570534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:10.570535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:10.570543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:10.572020 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.572512 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.572513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.572514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.572577 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.572853 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.572854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.572854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.572984 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:10.573058 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:10.573387 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 23:09:10.573480 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 23:09:10.573819 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.563555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41499 (local address 127.8.141.195:41451)
0809 23:09:10.563748 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:10.563796 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:10.564824 (+ 1028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:10.565133 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:10.565137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:10.565474 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:10.565686 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:10.565691 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.566438 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.566439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:10.567255 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:10.567257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.567656 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.567657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:10.567811 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.568510 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:10.568520 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:10.570422 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:10.572151 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:10.572161 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:10.572164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:10.572422 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:10.572644 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:10.572646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:10.572646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:10.572771 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:10.573060 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:10.573068 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:10.573222 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 23:09:10.573368 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 23:09:10.573552 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260809 23:09:10.574380 15723 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41499
I20260809 23:09:10.574481 15723 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:10.574688 15723 heartbeater.cc:511] Master 127.8.141.254:41499 requested a full tablet report, sending...
I20260809 23:09:10.575140 14931 ts_manager.cc:194] Registered new tserver with Master: 70b56a73e6ba464d99280ddc1734c021 (127.8.141.195:44119)
I20260809 23:09:10.575513 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:41451
I20260809 23:09:10.581223 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:09:10.609834 15732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.600542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44224 (local address 127.8.141.193:34749)
0809 23:09:10.600730 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:10.600736 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:10.600747 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:10.600776 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:10.600777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:10.600795 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:10.600878 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:10.601584 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.602383 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.603441 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.603797 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.603859 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.606356 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:10.606366 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:10.606366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:10.606373 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:10.608159 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.608699 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.608702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.608702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.608772 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.609045 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.609046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.609046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.609214 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:10.609465 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:10.609622 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 23:09:10.609703 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[14852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:10.618932 15006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:10.611327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51670 (local address 127.8.141.254:41499)
0809 23:09:10.611387 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:10.611389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:10.611406 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:09:10.611450 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:10.611451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:10.611460 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:10.611515 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:10.612015 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.612523 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.613185 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:10.613477 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:10.613555 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:10.616100 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:10.616101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:10.616102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:10.616139 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:10.617771 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.618150 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.618151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.618151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.618212 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:10.618428 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:10.618429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:10.618429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:10.618593 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:10.618674 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:10.618787 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 23:09:10.618857 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 23:09:10.619264 14931 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51670
I20260809 23:09:10.619688 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 15044
I20260809 23:09:10.636345 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 15270
I20260809 23:09:10.648998 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 15500
I20260809 23:09:10.663982 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 14876
2026-08-09T23:09:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2243 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[15738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[15738](info): set up 2 sockets
Aug 09 23:09:10 dist-test-slave-j2vl krb5kdc[15738](info): commencing operation
krb5kdc: starting...
W20260809 23:09:12.042887 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:12.189448 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:40555
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:40555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:12.281284 15766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:12.281558 15766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:12.281584 15766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:12.282759 15766 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:12.282797 15766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:12.282812 15766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:12.282824 15766 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:12.282836 15766 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:12.284375 15766 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:40555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:40555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:12.284595 15766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:12.284782 15766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:12.290361 15778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:12.290364 15775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:12.290376 15776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:12.291144 15766 server_base.cc:1034] running on GCE node
I20260809 23:09:12.291483 15766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:12.291844 15766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:12.293068 15766 hybrid_clock.cc:648] HybridClock initialized: now 1786316952293041 us; error 45 us; skew 500 ppm
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:12.295853 15766 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:12.296255 15766 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:12.296351 15766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:12.296511 15766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:12.297726 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "86f3b906e1e34264b51fe7551373c532"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "9f8cd36553f91e95122bb95842068f65"
server_key_iv: "3527041a31742d699c8195bf28777d40"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.298105 15766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "86f3b906e1e34264b51fe7551373c532"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "9f8cd36553f91e95122bb95842068f65"
server_key_iv: "3527041a31742d699c8195bf28777d40"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.299515 15766 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:12.300390 15791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.300688 15766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:12.300810 15766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "86f3b906e1e34264b51fe7551373c532"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "9f8cd36553f91e95122bb95842068f65"
server_key_iv: "3527041a31742d699c8195bf28777d40"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.300920 15766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:12.313133 15766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:12.315403 15766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:12.315608 15766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:12.319914 15766 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:40555
I20260809 23:09:12.319986 15875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:40555 every 8 connection(s)
I20260809 23:09:12.320322 15766 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:12.321696 15876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:12.324239 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532: Bootstrap starting.
I20260809 23:09:12.324822 15876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:12.325109 15876 log.cc:824] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:12.325829 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 15766
I20260809 23:09:12.325919 15876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532: No bootstrap required, opened a new log
I20260809 23:09:12.326000 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:12.326195 8759 external_mini_cluster.cc:1428] Setting key b5a6f94f79d334bf38019372682ca54f
I20260809 23:09:12.327970 15876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f3b906e1e34264b51fe7551373c532" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 40555 } }
I20260809 23:09:12.328163 15876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:12.328208 15876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86f3b906e1e34264b51fe7551373c532, State: Initialized, Role: FOLLOWER
I20260809 23:09:12.328402 15876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f3b906e1e34264b51fe7551373c532" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 40555 } }
I20260809 23:09:12.328507 15876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:12.328543 15876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:12.328626 15876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:12.329445 15876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f3b906e1e34264b51fe7551373c532" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 40555 } }
I20260809 23:09:12.329546 15876 leader_election.cc:302] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86f3b906e1e34264b51fe7551373c532; no voters:
I20260809 23:09:12.329777 15876 leader_election.cc:288] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:12.329818 15883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:12.329926 15883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [term 1 LEADER]: Becoming Leader. State: Replica: 86f3b906e1e34264b51fe7551373c532, State: Running, Role: LEADER
I20260809 23:09:12.330044 15883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f3b906e1e34264b51fe7551373c532" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 40555 } }
I20260809 23:09:12.330077 15876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:12.330394 15883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86f3b906e1e34264b51fe7551373c532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f3b906e1e34264b51fe7551373c532" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 40555 } } }
I20260809 23:09:12.330566 15883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:12.330770 15883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86f3b906e1e34264b51fe7551373c532. Latest consensus state: current_term: 1 leader_uuid: "86f3b906e1e34264b51fe7551373c532" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86f3b906e1e34264b51fe7551373c532" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 40555 } } }
I20260809 23:09:12.330878 15883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:12.331197 15897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:12.331841 15897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:12.333694 15897 catalog_manager.cc:1347] Generated new cluster ID: 38ab68b5cc2945d09f74a0328ac4d0a4
I20260809 23:09:12.333760 15897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:12.338661 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.326785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.8.141.254:40555)
0809 23:09:12.326975 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.326983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.327025 (+ 42us) server_negotiation.cc:407] Connection header received
0809 23:09:12.327945 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.327953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.328006 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.328140 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:12.329053 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.330429 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.331621 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.332281 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.332421 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.335196 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:12.335200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:12.335200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:12.335215 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:12.336883 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.337383 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.337387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.337388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.337516 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.337740 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.337741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.337742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.338013 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:12.338192 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.338265 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.338435 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260809 23:09:12.347829 15897 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:12.348453 15897 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:12.352694 15897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532: Generated new TSK 0
I20260809 23:09:12.352909 15897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:09:12.389221 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:40555
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:12.480353 15922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:12.480592 15922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:12.480620 15922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:12.481894 15922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:12.481945 15922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:12.481974 15922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:12.483541 15922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:40555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:12.483762 15922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:12.483958 15922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:12.489671 15934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:12.489856 15922 server_base.cc:1034] running on GCE node
W20260809 23:09:12.489707 15932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:12.489707 15931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:12.490195 15922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:12.490556 15922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:12.491730 15922 hybrid_clock.cc:648] HybridClock initialized: now 1786316952491712 us; error 31 us; skew 500 ppm
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:12.494777 15922 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:09:12.495208 15922 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:12.495273 15922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:12.495380 15922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:12.496480 15922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "b99affbf535e4f7bb8a9f4498de3b2e0"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "3189d89d68889941103204612cef1515"
server_key_iv: "678ba91be3d7108a67575cbb769ab8f1"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.496893 15922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "b99affbf535e4f7bb8a9f4498de3b2e0"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "3189d89d68889941103204612cef1515"
server_key_iv: "678ba91be3d7108a67575cbb769ab8f1"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.498555 15922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:12.499804 15943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.500077 15922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:12.500176 15922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "b99affbf535e4f7bb8a9f4498de3b2e0"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "3189d89d68889941103204612cef1515"
server_key_iv: "678ba91be3d7108a67575cbb769ab8f1"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.500243 15922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:12.512022 15922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:12.514706 15922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:12.514887 15922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:12.515116 15922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:12.515503 15922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:12.515547 15922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.515573 15922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:12.515610 15922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.523263 15922 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:34065
I20260809 23:09:12.523356 16120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:34065 every 8 connection(s)
I20260809 23:09:12.523649 15922 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:12.525671 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 15922
I20260809 23:09:12.525794 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:12.525993 8759 external_mini_cluster.cc:1428] Setting key 1ba3f2b742a2b36b3a182e4b06c53f3f
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
I20260809 23:09:12.536054 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.524634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:59189 (local address 127.8.141.254:40555)
0809 23:09:12.524743 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.524746 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.524915 (+ 169us) server_negotiation.cc:407] Connection header received
0809 23:09:12.525546 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.525547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.525558 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.525623 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:12.527187 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.527991 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.529265 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.529608 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.529695 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.532780 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:12.532782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:12.532783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:12.532792 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:12.534419 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.534934 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.534936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.534936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.535005 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.535290 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.535291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.535291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.535425 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:12.535511 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.535843 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.535950 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 23:09:12.536302 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.524565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:40555 (local address 127.8.141.193:59189)
0809 23:09:12.524745 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:12.524791 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:12.525310 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:12.525740 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:12.525745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:12.526165 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:12.527059 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.527066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.528134 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.528135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.529161 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.529165 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.529708 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.529709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.529861 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.530548 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:12.530561 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:12.532636 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:12.534549 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:12.534562 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:12.534565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:12.534847 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:12.535077 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:12.535078 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:12.535079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:12.535211 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:12.535503 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:12.535511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:12.535699 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 23:09:12.535820 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 23:09:12.536016 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260809 23:09:12.536906 16121 heartbeater.cc:347] Connected to a master server at 127.8.141.254:40555
I20260809 23:09:12.537045 16121 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:12.537226 16121 heartbeater.cc:511] Master 127.8.141.254:40555 requested a full tablet report, sending...
I20260809 23:09:12.538125 15817 ts_manager.cc:194] Registered new tserver with Master: b99affbf535e4f7bb8a9f4498de3b2e0 (127.8.141.193:34065)
I20260809 23:09:12.539070 15817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:59189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:09:12.564934 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:40555
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:12.656731 16129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:12.656981 16129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:12.657008 16129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:12.658595 16129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:12.658674 16129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:12.658708 16129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:12.660737 16129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:40555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:12.661010 16129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:12.661326 16129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:09:12.668726 16129 server_base.cc:1034] running on GCE node
W20260809 23:09:12.668648 16137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:12.668635 16139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:12.668673 16136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:12.669232 16129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:12.669579 16129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:12.670792 16129 hybrid_clock.cc:648] HybridClock initialized: now 1786316952670810 us; error 66 us; skew 500 ppm
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:12.674754 16129 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:09:12.675386 16129 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:12.675477 16129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:12.675657 16129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:12.676926 16129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "a9dc4f8558614c90aea944b01afd9233"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "59dbf8018845616593ae5cd4f4fc8c2b"
server_key_iv: "0c6ca18712f68fa04807b59aab8161c0"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.677443 16129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "a9dc4f8558614c90aea944b01afd9233"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "59dbf8018845616593ae5cd4f4fc8c2b"
server_key_iv: "0c6ca18712f68fa04807b59aab8161c0"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.679122 16129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:12.680226 16148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.680593 16129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:12.680734 16129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "a9dc4f8558614c90aea944b01afd9233"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "59dbf8018845616593ae5cd4f4fc8c2b"
server_key_iv: "0c6ca18712f68fa04807b59aab8161c0"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.680837 16129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:12.702529 16129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:12.705569 16129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:12.705705 16129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:12.706017 16129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:12.706369 16129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:12.706405 16129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.706430 16129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:12.706444 16129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.714397 16129 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:37387
I20260809 23:09:12.714680 16319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:37387 every 8 connection(s)
I20260809 23:09:12.714780 16129 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:12.722409 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 16129
I20260809 23:09:12.722520 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:12.722822 8759 external_mini_cluster.cc:1428] Setting key 73f1d22ba26f4b4fb98476feded6a601
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:12.727654 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.716122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:35005 (local address 127.8.141.254:40555)
0809 23:09:12.716207 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.716210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.716503 (+ 293us) server_negotiation.cc:407] Connection header received
0809 23:09:12.717068 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.717070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.717090 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.717222 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:12.719159 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.719696 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.720864 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.721070 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.721144 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.724262 (+ 3118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:12.724264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:12.724264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:12.724275 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:12.725895 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.726519 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.726521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.726522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.726595 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.726911 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.726912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.726912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.727047 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:12.727125 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.727447 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.727554 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:09:12.727875 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.716073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:40555 (local address 127.8.141.194:35005)
0809 23:09:12.716345 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:12.716390 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:12.716899 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:12.717637 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:12.717641 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:12.717968 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:12.718953 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.718961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.719830 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.719831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.720712 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.720715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.721187 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.721188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.721404 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.722206 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:12.722217 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:12.724053 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:12.725992 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:12.726003 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:12.726007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:12.726401 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:12.726688 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:12.726688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:12.726689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:12.726827 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:12.727126 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:12.727135 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:12.727297 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 23:09:12.727411 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:09:12.727600 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":148,"threads_started":1}
I20260809 23:09:12.728480 16321 heartbeater.cc:347] Connected to a master server at 127.8.141.254:40555
I20260809 23:09:12.728590 16321 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:12.728792 16321 heartbeater.cc:511] Master 127.8.141.254:40555 requested a full tablet report, sending...
I20260809 23:09:12.729332 15817 ts_manager.cc:194] Registered new tserver with Master: a9dc4f8558614c90aea944b01afd9233 (127.8.141.194:37387)
I20260809 23:09:12.730018 15817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:35005
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:09:12.757405 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:40555
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:12.848235 16326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:12.848480 16326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:12.848510 16326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:12.849977 16326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:12.850039 16326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:12.850073 16326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:12.851856 16326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:40555
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:12.852164 16326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:12.852490 16326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:12.859324 16340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:12.859454 16339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:12.859463 16326 server_base.cc:1034] running on GCE node
W20260809 23:09:12.859378 16343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:12.859835 16326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:12.860124 16326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:12.861303 16326 hybrid_clock.cc:648] HybridClock initialized: now 1786316952861293 us; error 55 us; skew 500 ppm
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:12.864468 16326 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:09:12.864917 16326 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:12.864972 16326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:12.865113 16326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:12.866340 16326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "f1f56acc9a3c421c9b8304c7b7be4db3"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "9eee07a2991afbc8a0205b3e04e4ce75"
server_key_iv: "95aaea25915df866dfc22c6d5034f3a3"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.866736 16326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "f1f56acc9a3c421c9b8304c7b7be4db3"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "9eee07a2991afbc8a0205b3e04e4ce75"
server_key_iv: "95aaea25915df866dfc22c6d5034f3a3"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.868237 16326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:12.869226 16353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.869589 16326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:12.869693 16326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "f1f56acc9a3c421c9b8304c7b7be4db3"
format_stamp: "Formatted at 2026-08-09 23:09:12 on dist-test-slave-j2vl"
server_key: "9eee07a2991afbc8a0205b3e04e4ce75"
server_key_iv: "95aaea25915df866dfc22c6d5034f3a3"
server_key_version: "encryptionkey@0"
I20260809 23:09:12.869828 16326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:12.882805 16326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:12.885692 16326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:12.885831 16326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:12.886154 16326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:12.886538 16326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:12.886597 16326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.886636 16326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:12.886683 16326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:12.893599 16326 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:42613
I20260809 23:09:12.893671 16543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:42613 every 8 connection(s)
I20260809 23:09:12.894022 16326 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:09:12 dist-test-slave-j2vl krb5kdc[15738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316952, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:12.904181 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 16326
I20260809 23:09:12.904352 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:12.904453 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.894985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:51867 (local address 127.8.141.254:40555)
0809 23:09:12.895076 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.895078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.895204 (+ 126us) server_negotiation.cc:407] Connection header received
0809 23:09:12.895707 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.895708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.895721 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.895792 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:12.896868 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.897421 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.898625 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.898830 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.898902 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.901445 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:12.901447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:12.901447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:12.901455 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:12.902868 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.903338 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.903340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.903340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.903414 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.903707 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.903708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.903708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.903871 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:12.903934 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.904248 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.904342 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 23:09:12.904627 8759 external_mini_cluster.cc:1428] Setting key b4c42d88b330d1e28a0a71142ecee45f
I20260809 23:09:12.904771 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.894882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:40555 (local address 127.8.141.195:51867)
0809 23:09:12.895076 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:12.895122 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:12.895611 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:12.895893 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:12.895897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:12.896193 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:12.896720 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.896726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.897583 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.897584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.898511 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.898514 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.898942 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.898944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.899065 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.899699 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:12.899709 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:12.901262 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:12.902971 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:12.902980 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:12.902983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:12.903254 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:12.903493 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:12.903494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:12.903494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:12.903629 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:12.903934 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:12.903943 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:12.904111 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 23:09:12.904224 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 23:09:12.904508 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":106,"threads_started":1}
I20260809 23:09:12.905333 16544 heartbeater.cc:347] Connected to a master server at 127.8.141.254:40555
I20260809 23:09:12.905491 16544 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:12.905663 16544 heartbeater.cc:511] Master 127.8.141.254:40555 requested a full tablet report, sending...
I20260809 23:09:12.906069 15816 ts_manager.cc:194] Registered new tserver with Master: f1f56acc9a3c421c9b8304c7b7be4db3 (127.8.141.195:42613)
I20260809 23:09:12.906476 15816 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:51867
I20260809 23:09:12.917294 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:09:12.925609 15882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.918787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.8.141.254:40555)
0809 23:09:12.918853 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.918856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.918996 (+ 140us) server_negotiation.cc:407] Connection header received
0809 23:09:12.919088 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.919089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.919101 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.919153 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:12.919746 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.920253 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.920975 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.921177 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.921274 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.922681 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:12.922683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:12.922683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:12.922699 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:12.924073 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.924471 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.924472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.924472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.924537 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:12.924747 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:12.924748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:12.924749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:12.925015 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:12.925190 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.925388 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.925488 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:09:12.927475 15816 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:09:12.929131 15816 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:12.936620 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.932032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42613 (local address 127.0.0.1:44306)
0809 23:09:12.932815 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:12.932824 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:12.932874 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:12.933134 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:12.933135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:12.933137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:12.933180 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.933182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.934563 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.934564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.935520 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.935523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.936257 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.936259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.936355 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.936372 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 23:09:12.936445 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:09:12.936501 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":63,"thread_start_us":74,"threads_started":1}
I20260809 23:09:12.936815 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.932043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:34065 (local address 127.0.0.1:56690)
0809 23:09:12.932945 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:12.932954 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:12.933035 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:12.933225 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:12.933226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:12.933228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:12.933279 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.933280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.934165 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.934166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.935528 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.935532 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.936558 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.936559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.936624 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.936628 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:12.936679 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:09:12.936722 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
I20260809 23:09:12.937037 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.932237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.8.141.195:42613)
0809 23:09:12.932431 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.932437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.932881 (+ 444us) server_negotiation.cc:407] Connection header received
0809 23:09:12.932938 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.932940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.932956 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.933038 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:12.933269 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.934363 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.935624 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.936167 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.936266 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.936334 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.936895 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.936943 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260809 23:09:12.937119 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.932255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56690 (local address 127.8.141.193:34065)
0809 23:09:12.932482 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.932488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.933008 (+ 520us) server_negotiation.cc:407] Connection header received
0809 23:09:12.933109 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.933111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.933127 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.933225 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:12.933358 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.934059 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.935642 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.936368 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.936432 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.936491 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.936705 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.937024 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260809 23:09:12.937309 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.932031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37387 (local address 127.0.0.1:49898)
0809 23:09:12.932553 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:12.932603 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:12.932702 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:12.932919 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:12.932923 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:12.932928 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:12.933027 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.933030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.934423 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.934424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.935530 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:12.935532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.936461 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.936462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:12.936535 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.936539 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:12.936593 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:09:12.936627 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":4,"thread_start_us":127,"threads_started":1}
I20260809 23:09:12.937808 16018 tablet_service.cc:1467] Processing CreateTablet for tablet 032b970e205446029bb47b7a6cffe4aa (DEFAULT_TABLE table=test-table [id=5826991c75614f39a3fdbf02116604c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:12.937812 16432 tablet_service.cc:1467] Processing CreateTablet for tablet 032b970e205446029bb47b7a6cffe4aa (DEFAULT_TABLE table=test-table [id=5826991c75614f39a3fdbf02116604c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:12.938122 16018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 032b970e205446029bb47b7a6cffe4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:12.938172 16432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 032b970e205446029bb47b7a6cffe4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:12.940989 16568 tablet_bootstrap.cc:492] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Bootstrap starting.
I20260809 23:09:12.940999 16569 tablet_bootstrap.cc:492] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: Bootstrap starting.
I20260809 23:09:12.941200 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:12.932324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49898 (local address 127.8.141.194:37387)
0809 23:09:12.932513 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:12.932517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:12.932678 (+ 161us) server_negotiation.cc:407] Connection header received
0809 23:09:12.932791 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:12.932794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:12.932811 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:12.932901 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:12.933121 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.934310 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.935636 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:12.936376 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:12.936447 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:12.936497 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:12.936605 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 23:09:12.936659 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260809 23:09:12.941830 16569 tablet_bootstrap.cc:654] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:12.941856 16568 tablet_bootstrap.cc:654] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:12.941994 16221 tablet_service.cc:1467] Processing CreateTablet for tablet 032b970e205446029bb47b7a6cffe4aa (DEFAULT_TABLE table=test-table [id=5826991c75614f39a3fdbf02116604c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:12.942408 16221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 032b970e205446029bb47b7a6cffe4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:12.942476 16569 log.cc:824] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:12.942451 16568 log.cc:824] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:12.943316 16568 tablet_bootstrap.cc:492] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: No bootstrap required, opened a new log
I20260809 23:09:12.943389 16568 ts_tablet_manager.cc:1397] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:09:12.945082 16568 raft_consensus.cc:348] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:12.945276 16568 raft_consensus.cc:374] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:12.945327 16568 raft_consensus.cc:729] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1f56acc9a3c421c9b8304c7b7be4db3, State: Initialized, Role: FOLLOWER
I20260809 23:09:12.945264 16573 tablet_bootstrap.cc:492] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Bootstrap starting.
I20260809 23:09:12.945521 16568 consensus_queue.cc:260] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:12.945959 16544 heartbeater.cc:503] Master 127.8.141.254:40555 was elected leader, sending a full tablet report...
I20260809 23:09:12.945972 16568 ts_tablet_manager.cc:1428] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 23:09:12.946154 16573 tablet_bootstrap.cc:654] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:12.946156 16569 tablet_bootstrap.cc:492] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: No bootstrap required, opened a new log
I20260809 23:09:12.946224 16569 ts_tablet_manager.cc:1397] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260809 23:09:12.946797 16573 log.cc:824] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:12.947649 16573 tablet_bootstrap.cc:492] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: No bootstrap required, opened a new log
I20260809 23:09:12.947726 16573 ts_tablet_manager.cc:1397] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:09:12.948514 16569 raft_consensus.cc:348] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:12.948700 16569 raft_consensus.cc:374] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:12.948740 16569 raft_consensus.cc:729] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b99affbf535e4f7bb8a9f4498de3b2e0, State: Initialized, Role: FOLLOWER
I20260809 23:09:12.948874 16569 consensus_queue.cc:260] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:12.949183 16569 ts_tablet_manager.cc:1428] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:09:12.949204 16121 heartbeater.cc:503] Master 127.8.141.254:40555 was elected leader, sending a full tablet report...
I20260809 23:09:12.949928 16573 raft_consensus.cc:348] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:12.950122 16573 raft_consensus.cc:374] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:12.950160 16573 raft_consensus.cc:729] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9dc4f8558614c90aea944b01afd9233, State: Initialized, Role: FOLLOWER
I20260809 23:09:12.950289 16573 consensus_queue.cc:260] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:12.950618 16573 ts_tablet_manager.cc:1428] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:09:12.950698 16321 heartbeater.cc:503] Master 127.8.141.254:40555 was elected leader, sending a full tablet report...
W20260809 23:09:12.965606 16322 tablet.cc:2369] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:09:13.024529 16123 tablet.cc:2369] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:09:13.036132 16574 raft_consensus.cc:482] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:13.036290 16574 raft_consensus.cc:504] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:13.036803 16574 leader_election.cc:288] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b99affbf535e4f7bb8a9f4498de3b2e0 (127.8.141.193:34065), a9dc4f8558614c90aea944b01afd9233 (127.8.141.194:37387)
I20260809 23:09:13.041069 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.037005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:34065 (local address 127.8.141.195:43415)
0809 23:09:13.037105 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.037119 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.037210 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.037431 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.037432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.037434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:13.037529 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:13.037532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.038516 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.038518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:13.039874 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:13.039879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.040747 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.040749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:13.040846 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:13.040848 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.040909 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.040959 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":31}
I20260809 23:09:13.041113 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.037068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:43415 (local address 127.8.141.193:34065)
0809 23:09:13.037136 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.037139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.037214 (+ 75us) server_negotiation.cc:407] Connection header received
0809 23:09:13.037298 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.037299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.037311 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.037419 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:13.037621 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.038403 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.040026 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.040631 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.040720 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:13.040730 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.040964 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.041035 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:09:13.041113 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.037265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:51679 (local address 127.8.141.194:37387)
0809 23:09:13.037314 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.037319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.037586 (+ 267us) server_negotiation.cc:407] Connection header received
0809 23:09:13.037623 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.037624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.037634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.037688 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:13.037856 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.038543 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.040026 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.040631 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.040720 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:13.040730 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.040990 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.041048 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:09:13.041079 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.037226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37387 (local address 127.8.141.195:51679)
0809 23:09:13.037478 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.037491 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.037563 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.037697 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.037697 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.037699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:13.037771 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:13.037772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.038635 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.038636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:13.039874 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:13.039879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.040748 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.040749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:13.040835 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:13.040840 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.040934 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.040963 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":39,"thread_start_us":172,"threads_started":1}
I20260809 23:09:13.041531 16048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "032b970e205446029bb47b7a6cffe4aa" candidate_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" is_pre_election: true
I20260809 23:09:13.041611 16252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "032b970e205446029bb47b7a6cffe4aa" candidate_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9dc4f8558614c90aea944b01afd9233" is_pre_election: true
I20260809 23:09:13.041734 16048 raft_consensus.cc:2454] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1f56acc9a3c421c9b8304c7b7be4db3 in term 0.
I20260809 23:09:13.041778 16252 raft_consensus.cc:2454] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1f56acc9a3c421c9b8304c7b7be4db3 in term 0.
I20260809 23:09:13.042001 16356 leader_election.cc:302] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b99affbf535e4f7bb8a9f4498de3b2e0, f1f56acc9a3c421c9b8304c7b7be4db3; no voters:
I20260809 23:09:13.042182 16574 raft_consensus.cc:2781] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:13.042260 16574 raft_consensus.cc:482] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:13.042280 16574 raft_consensus.cc:3037] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:13.043082 16574 raft_consensus.cc:504] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:13.043257 16574 leader_election.cc:288] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [CANDIDATE]: Term 1 election: Requested vote from peers b99affbf535e4f7bb8a9f4498de3b2e0 (127.8.141.193:34065), a9dc4f8558614c90aea944b01afd9233 (127.8.141.194:37387)
I20260809 23:09:13.043431 16048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "032b970e205446029bb47b7a6cffe4aa" candidate_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0"
I20260809 23:09:13.043438 16252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "032b970e205446029bb47b7a6cffe4aa" candidate_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9dc4f8558614c90aea944b01afd9233"
I20260809 23:09:13.043517 16048 raft_consensus.cc:3037] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:13.043553 16252 raft_consensus.cc:3037] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:13.044210 16048 raft_consensus.cc:2454] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1f56acc9a3c421c9b8304c7b7be4db3 in term 1.
I20260809 23:09:13.044415 16252 raft_consensus.cc:2454] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1f56acc9a3c421c9b8304c7b7be4db3 in term 1.
I20260809 23:09:13.044404 16356 leader_election.cc:302] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b99affbf535e4f7bb8a9f4498de3b2e0, f1f56acc9a3c421c9b8304c7b7be4db3; no voters:
I20260809 23:09:13.044678 16574 raft_consensus.cc:2781] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:13.044927 16574 raft_consensus.cc:686] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 1 LEADER]: Becoming Leader. State: Replica: f1f56acc9a3c421c9b8304c7b7be4db3, State: Running, Role: LEADER
I20260809 23:09:13.045178 16574 consensus_queue.cc:237] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } }
I20260809 23:09:13.047045 15816 catalog_manager.cc:5495] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1f56acc9a3c421c9b8304c7b7be4db3 (127.8.141.195). New cstate: current_term: 1 leader_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1f56acc9a3c421c9b8304c7b7be4db3" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 } health_report { overall_health: UNKNOWN } } }
I20260809 23:09:13.053510 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.050728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44318 (local address 127.8.141.195:42613)
0809 23:09:13.050801 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.050803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.050813 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:09:13.050858 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.050859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.050887 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.050974 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:13.051143 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.051688 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.052549 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:13.052766 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:13.052852 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:13.053065 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:13.053180 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:13.053283 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.053334 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.053383 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 23:09:13.055749 16252 raft_consensus.cc:1261] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 1 FOLLOWER]: Refusing update from remote peer f1f56acc9a3c421c9b8304c7b7be4db3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:13.056185 16048 raft_consensus.cc:1261] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0 [term 1 FOLLOWER]: Refusing update from remote peer f1f56acc9a3c421c9b8304c7b7be4db3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:13.056214 16574 consensus_queue.cc:1035] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9dc4f8558614c90aea944b01afd9233" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:13.056561 16574 consensus_queue.cc:1035] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b99affbf535e4f7bb8a9f4498de3b2e0" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 34065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:13.059592 16589 mvcc.cc:205] Tried to move back new op lower bound from 7316754239712821248 to 7316754239675117568. Current Snapshot: MvccSnapshot[applied={T|T < 7316754239712821248}]
I20260809 23:09:13.059620 16590 mvcc.cc:205] Tried to move back new op lower bound from 7316754239712821248 to 7316754239675117568. Current Snapshot: MvccSnapshot[applied={T|T < 7316754239712821248}]
I20260809 23:09:13.062101 15817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:13.062224 15817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:13.063084 15817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86f3b906e1e34264b51fe7551373c532: Sending DeleteTablet for 3 replicas of tablet 032b970e205446029bb47b7a6cffe4aa
I20260809 23:09:13.063563 16018 tablet_service.cc:1514] Processing DeleteTablet for tablet 032b970e205446029bb47b7a6cffe4aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:13 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:56690
I20260809 23:09:13.063696 16432 tablet_service.cc:1514] Processing DeleteTablet for tablet 032b970e205446029bb47b7a6cffe4aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:13 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:44306
I20260809 23:09:13.063725 16221 tablet_service.cc:1514] Processing DeleteTablet for tablet 032b970e205446029bb47b7a6cffe4aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:13 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:49898
I20260809 23:09:13.064059 16605 tablet_replica.cc:330] T 032b970e205446029bb47b7a6cffe4aa P b99affbf535e4f7bb8a9f4498de3b2e0: stopping tablet replica
I20260809 23:09:13.064194 16606 tablet_replica.cc:330] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: stopping tablet replica
I20260809 23:09:13.064311 16606 raft_consensus.cc:2229] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:13.064380 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 15922
I20260809 23:09:13.064419 16606 raft_consensus.cc:2258] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:13.065122 16608 tablet_replica.cc:330] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: stopping tablet replica
I20260809 23:09:13.065248 16608 raft_consensus.cc:2229] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:13.065433 16608 raft_consensus.cc:2258] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:13.074553 16606 ts_tablet_manager.cc:1905] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:13.078464 16608 ts_tablet_manager.cc:1905] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:13.083350 16608 ts_tablet_manager.cc:1918] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:13.084664 16608 log.cc:1196] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/032b970e205446029bb47b7a6cffe4aa
I20260809 23:09:13.085202 16608 ts_tablet_manager.cc:1939] T 032b970e205446029bb47b7a6cffe4aa P a9dc4f8558614c90aea944b01afd9233: Deleting consensus metadata
I20260809 23:09:13.086150 15798 catalog_manager.cc:4841] TS a9dc4f8558614c90aea944b01afd9233 (127.8.141.194:37387): tablet 032b970e205446029bb47b7a6cffe4aa (table test-table [id=5826991c75614f39a3fdbf02116604c9]) successfully deleted
I20260809 23:09:13.092656 16606 ts_tablet_manager.cc:1918] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:13.092790 16606 log.cc:1196] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/032b970e205446029bb47b7a6cffe4aa
I20260809 23:09:13.093179 16606 ts_tablet_manager.cc:1939] T 032b970e205446029bb47b7a6cffe4aa P f1f56acc9a3c421c9b8304c7b7be4db3: Deleting consensus metadata
I20260809 23:09:13.093792 15802 catalog_manager.cc:4841] TS f1f56acc9a3c421c9b8304c7b7be4db3 (127.8.141.195:42613): tablet 032b970e205446029bb47b7a6cffe4aa (table test-table [id=5826991c75614f39a3fdbf02116604c9]) successfully deleted
W20260809 23:09:13.097997 15800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:34065 (error 108)
I20260809 23:09:13.098278 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 16129
I20260809 23:09:13.101275 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.098465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:34065 (local address 127.0.0.1:56702)
0809 23:09:13.101139 (+ 2674us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.101212 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:34065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2629}
W20260809 23:09:13.109275 15800 catalog_manager.cc:4584] TS b99affbf535e4f7bb8a9f4498de3b2e0 (127.8.141.193:34065): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 032b970e205446029bb47b7a6cffe4aa: Network error: Client connection negotiation failed: client connection to 127.8.141.193:34065: connect: Connection refused (error 111)
I20260809 23:09:13.119073 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 16326
I20260809 23:09:13.133318 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 15766
2026-08-09T23:09:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2469 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T23:09:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:13Z Disabled control of system clock
I20260809 23:09:13.177223 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:43877
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:43877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 23:09:13.264909 16620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:13.265156 16620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:13.265180 16620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:13.266395 16620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:13.266444 16620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:13.266458 16620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:13.266470 16620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:13.266484 16620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:13.267874 16620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:43877
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:43877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:13.268111 16620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:13.268384 16620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:13.274748 16627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.274685 16637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.274685 16628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:13.275828 16620 server_base.cc:1034] running on GCE node
I20260809 23:09:13.276147 16620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:13.276520 16620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:13.277695 16620 hybrid_clock.cc:648] HybridClock initialized: now 1786316953277683 us; error 36 us; skew 500 ppm
I20260809 23:09:13.279258 16620 webserver.cc:460] Webserver started at http://127.8.141.254:46705/ using document root <none> and password file <none>
I20260809 23:09:13.279580 16620 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:13.279661 16620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:13.279826 16620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:13.281039 16620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "f54fbc65ad8625428a6471bb16461d94"
server_key_iv: "7b64d7a08734a4168c3ca9d46c2ed4ff"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.281525 16620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "f54fbc65ad8625428a6471bb16461d94"
server_key_iv: "7b64d7a08734a4168c3ca9d46c2ed4ff"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.283282 16620 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 23:09:13.284376 16644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.284660 16620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:13.284788 16620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "f54fbc65ad8625428a6471bb16461d94"
server_key_iv: "7b64d7a08734a4168c3ca9d46c2ed4ff"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.284868 16620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:13.288772 16620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:13.289108 16620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:13.289268 16620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:13.294085 16620 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:43877
I20260809 23:09:13.294130 16725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:43877 every 8 connection(s)
I20260809 23:09:13.294628 16620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:13.296394 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:13.299484 16726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3: Bootstrap starting.
I20260809 23:09:13.300073 16726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:13.300328 16726 log.cc:824] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:13.301071 16726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3: No bootstrap required, opened a new log
I20260809 23:09:13.302767 16726 raft_consensus.cc:348] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43877 } }
I20260809 23:09:13.302888 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 16620
I20260809 23:09:13.302933 16726 raft_consensus.cc:374] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:13.302958 16726 raft_consensus.cc:729] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cab6f5ef0c0940f1a9d64a0eb62c2dc3, State: Initialized, Role: FOLLOWER
I20260809 23:09:13.303014 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:13.303160 16726 consensus_queue.cc:260] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43877 } }
I20260809 23:09:13.303203 8759 external_mini_cluster.cc:1428] Setting key df65964f87ac0f68a04e5b913c6c37be
I20260809 23:09:13.303246 16726 raft_consensus.cc:388] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:13.303272 16726 raft_consensus.cc:482] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:13.303350 16726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:13.304169 16726 raft_consensus.cc:504] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43877 } }
I20260809 23:09:13.304303 16726 leader_election.cc:302] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cab6f5ef0c0940f1a9d64a0eb62c2dc3; no voters:
I20260809 23:09:13.304471 16726 leader_election.cc:288] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:13.304927 16726 sys_catalog.cc:564] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:13.304509 16734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:13.305212 16734 raft_consensus.cc:686] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [term 1 LEADER]: Becoming Leader. State: Replica: cab6f5ef0c0940f1a9d64a0eb62c2dc3, State: Running, Role: LEADER
I20260809 23:09:13.305485 16734 consensus_queue.cc:237] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43877 } }
I20260809 23:09:13.306056 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.303796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58612 (local address 127.8.141.254:43877)
0809 23:09:13.304024 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.304032 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.304058 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:09:13.304156 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.304162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.304197 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.304346 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.305272 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.305276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.305277 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.305281 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.305403 (+ 122us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.305423 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.305514 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.305771 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.305840 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260809 23:09:13.306577 16737 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cab6f5ef0c0940f1a9d64a0eb62c2dc3. Latest consensus state: current_term: 1 leader_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43877 } } }
I20260809 23:09:13.306684 16737 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:13.306897 16756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:13.306906 16735 sys_catalog.cc:455] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cab6f5ef0c0940f1a9d64a0eb62c2dc3" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43877 } } }
I20260809 23:09:13.307472 16735 sys_catalog.cc:458] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:13.308141 16756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:13.309945 16756 catalog_manager.cc:1347] Generated new cluster ID: 6feabc59051f41ba93bd4ff502dd2ebc
I20260809 23:09:13.310012 16756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:13.317196 16756 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:13.317430 16756 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:13.326350 16756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3: Generated new TSK 0
I20260809 23:09:13.326665 16756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:09:13.328676 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:43877
--builtin_ntp_servers=127.8.141.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 23:09:13.423749 16763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:13.424000 16763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:13.424026 16763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:13.425494 16763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:13.425556 16763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:13.425587 16763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:13.427229 16763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:43877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:13.427520 16763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:13.427878 16763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:13.433523 16776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.433553 16778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.433553 16781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:13.434613 16763 server_base.cc:1034] running on GCE node
I20260809 23:09:13.434901 16763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:13.435323 16763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:13.436532 16763 hybrid_clock.cc:648] HybridClock initialized: now 1786316953436502 us; error 51 us; skew 500 ppm
I20260809 23:09:13.438548 16763 webserver.cc:460] Webserver started at http://127.8.141.193:39397/ using document root <none> and password file <none>
I20260809 23:09:13.438866 16763 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:13.438949 16763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:13.439103 16763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:13.440752 16763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "81eea906a7004251abd96a9819f0d556"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "970aa17b816de85804cf7b7fcd5f1290"
server_key_iv: "6bf518d11e97989b6951dd766933447d"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.441334 16763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "81eea906a7004251abd96a9819f0d556"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "970aa17b816de85804cf7b7fcd5f1290"
server_key_iv: "6bf518d11e97989b6951dd766933447d"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.443501 16763 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 23:09:13.444815 16794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.445214 16763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:13.445374 16763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "81eea906a7004251abd96a9819f0d556"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "970aa17b816de85804cf7b7fcd5f1290"
server_key_iv: "6bf518d11e97989b6951dd766933447d"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.445483 16763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:13.450879 16763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:13.451210 16763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:13.451355 16763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:13.451578 16763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:13.451933 16763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:13.451977 16763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.452006 16763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:13.452023 16763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.459363 16763 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:42997
I20260809 23:09:13.459443 16981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:42997 every 8 connection(s)
I20260809 23:09:13.459894 16763 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:13.464237 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.460745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:38043 (local address 127.8.141.254:43877)
0809 23:09:13.460820 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.460823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.461108 (+ 285us) server_negotiation.cc:407] Connection header received
0809 23:09:13.461749 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.461751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.461760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.461838 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.463922 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.463924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.463925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.463932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.463956 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.463963 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.464061 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.464134 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.464155 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 23:09:13.464402 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.460729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:43877 (local address 127.8.141.193:38043)
0809 23:09:13.460952 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.460997 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.461629 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.461910 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.461914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.461922 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:13.463719 (+ 1797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:13.463720 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:13.463754 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:13.463789 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:13.464055 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:13.464058 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:13.464061 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.464120 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.464175 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":115,"threads_started":1}
I20260809 23:09:13.465160 16983 heartbeater.cc:347] Connected to a master server at 127.8.141.254:43877
I20260809 23:09:13.465276 16983 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:13.465489 16983 heartbeater.cc:511] Master 127.8.141.254:43877 requested a full tablet report, sending...
I20260809 23:09:13.466091 16672 ts_manager.cc:194] Registered new tserver with Master: 81eea906a7004251abd96a9819f0d556 (127.8.141.193:42997)
I20260809 23:09:13.467687 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 16763
I20260809 23:09:13.467810 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:13.468012 8759 external_mini_cluster.cc:1428] Setting key bd208b51ab47c2722ee55155e77538ba
I20260809 23:09:13.469130 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:43877
--builtin_ntp_servers=127.8.141.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 23:09:13.560472 16987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:13.560753 16987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:13.560792 16987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:13.562125 16987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:13.562202 16987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:13.562247 16987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:13.563784 16987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:43877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:13.564096 16987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:13.564451 16987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:13.570366 16996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.570405 16993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.570390 16994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:13.570822 16987 server_base.cc:1034] running on GCE node
I20260809 23:09:13.571070 16987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:13.571312 16987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:13.572464 16987 hybrid_clock.cc:648] HybridClock initialized: now 1786316953572457 us; error 38 us; skew 500 ppm
I20260809 23:09:13.573954 16987 webserver.cc:460] Webserver started at http://127.8.141.194:46865/ using document root <none> and password file <none>
I20260809 23:09:13.574245 16987 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:13.574302 16987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:13.574448 16987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:13.575727 16987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "c55e716f83f74025b614182b3ea0a30b"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "7bfde543d573eb7b388e55b709604ee8"
server_key_iv: "0edd784afe4dcbc78df67ef61ed78cd5"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.576222 16987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "c55e716f83f74025b614182b3ea0a30b"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "7bfde543d573eb7b388e55b709604ee8"
server_key_iv: "0edd784afe4dcbc78df67ef61ed78cd5"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.577804 16987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 23:09:13.578788 17004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.579080 16987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:13.579238 16987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "c55e716f83f74025b614182b3ea0a30b"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "7bfde543d573eb7b388e55b709604ee8"
server_key_iv: "0edd784afe4dcbc78df67ef61ed78cd5"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.579334 16987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:13.583346 16987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:13.583663 16987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:13.583820 16987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:13.584080 16987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:13.584434 16987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:13.584496 16987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.584545 16987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:13.584573 16987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.591739 16987 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:36651
I20260809 23:09:13.591790 17187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:36651 every 8 connection(s)
I20260809 23:09:13.592178 16987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:13.595530 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.592982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49059 (local address 127.8.141.254:43877)
0809 23:09:13.593085 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.593088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.593308 (+ 220us) server_negotiation.cc:407] Connection header received
0809 23:09:13.593904 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.593906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.593912 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.593971 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.595243 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.595246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.595246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.595251 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.595269 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.595274 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.595363 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.595436 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.595452 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:09:13.595757 17195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.592907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:43877 (local address 127.8.141.194:49059)
0809 23:09:13.593173 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.593211 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.593781 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.594046 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.594050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.594060 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:13.595080 (+ 1020us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:13.595080 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:13.595111 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:13.595143 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:13.595350 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:13.595358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:13.595361 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.595449 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.595518 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
I20260809 23:09:13.596021 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 16987
I20260809 23:09:13.596141 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:13.596247 17190 heartbeater.cc:347] Connected to a master server at 127.8.141.254:43877
I20260809 23:09:13.596344 8759 external_mini_cluster.cc:1428] Setting key 51d7cf69ff59c15112a47f9d234a64c2
I20260809 23:09:13.596381 17190 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:13.596561 17190 heartbeater.cc:511] Master 127.8.141.254:43877 requested a full tablet report, sending...
I20260809 23:09:13.597000 16672 ts_manager.cc:194] Registered new tserver with Master: c55e716f83f74025b614182b3ea0a30b (127.8.141.194:36651)
I20260809 23:09:13.597395 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:43877
--builtin_ntp_servers=127.8.141.212:42539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 23:09:13.686244 17196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:13.686467 17196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:13.686492 17196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:13.687695 17196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:13.687734 17196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:13.687762 17196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:13.689106 17196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:43877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:13.689332 17196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:13.689689 17196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:13.695123 17203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.695137 17209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:13.695173 17205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:13.695504 17196 server_base.cc:1034] running on GCE node
I20260809 23:09:13.695792 17196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:13.696055 17196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:13.697242 17196 hybrid_clock.cc:648] HybridClock initialized: now 1786316953697227 us; error 31 us; skew 500 ppm
I20260809 23:09:13.698747 17196 webserver.cc:460] Webserver started at http://127.8.141.195:45497/ using document root <none> and password file <none>
I20260809 23:09:13.699014 17196 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:13.699091 17196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:13.699258 17196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:13.700379 17196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "1bb5324291c648acbe44711281822808"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "f2d1f798924f027e5d38e9272b731753"
server_key_iv: "afcb8c975fb9ab8206aab87082cbef0e"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.700765 17196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "1bb5324291c648acbe44711281822808"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "f2d1f798924f027e5d38e9272b731753"
server_key_iv: "afcb8c975fb9ab8206aab87082cbef0e"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.702299 17196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 23:09:13.703195 17221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.703431 17196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:13.703648 17196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "1bb5324291c648acbe44711281822808"
format_stamp: "Formatted at 2026-08-09 23:09:13 on dist-test-slave-j2vl"
server_key: "f2d1f798924f027e5d38e9272b731753"
server_key_iv: "afcb8c975fb9ab8206aab87082cbef0e"
server_key_version: "encryptionkey@0"
I20260809 23:09:13.703825 17196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:13.707836 17196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:13.708134 17196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:13.708288 17196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:13.708525 17196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:13.708868 17196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:13.708925 17196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.708966 17196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:13.708994 17196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:13.715418 17196 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:44853
I20260809 23:09:13.715481 17394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:44853 every 8 connection(s)
I20260809 23:09:13.715895 17196 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:13.720006 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.716978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:53613 (local address 127.8.141.254:43877)
0809 23:09:13.717030 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.717032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.717255 (+ 223us) server_negotiation.cc:407] Connection header received
0809 23:09:13.717831 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.717832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.717839 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.717914 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.719662 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.719664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.719664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.719669 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.719687 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.719691 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.719757 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.719896 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.719915 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:13.720329 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.716918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:43877 (local address 127.8.141.195:53613)
0809 23:09:13.717117 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.717164 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.717697 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.718066 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.718070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.718079 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:13.719429 (+ 1350us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:13.719430 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:13.719477 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:13.719530 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:13.719772 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:13.719779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:13.719782 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.719858 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.719930 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260809 23:09:13.720958 17395 heartbeater.cc:347] Connected to a master server at 127.8.141.254:43877
I20260809 23:09:13.721068 17395 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:13.721199 17395 heartbeater.cc:511] Master 127.8.141.254:43877 requested a full tablet report, sending...
I20260809 23:09:13.721683 16672 ts_manager.cc:194] Registered new tserver with Master: 1bb5324291c648acbe44711281822808 (127.8.141.195:44853)
I20260809 23:09:13.723570 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 17196
I20260809 23:09:13.723660 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:13.723869 8759 external_mini_cluster.cc:1428] Setting key d8fbddb2b86528547712c30d01593d79
I20260809 23:09:13.724927 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:09:13.727725 16733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.726440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.8.141.254:43877)
0809 23:09:13.726503 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.726505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.726663 (+ 158us) server_negotiation.cc:407] Connection header received
0809 23:09:13.726695 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.726696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.726701 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.726760 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.727408 (+ 648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.727409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.727409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.727413 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.727431 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.727435 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.727494 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.727590 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.727603 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:09:13.729650 16672 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:09:13.731356 16672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:13.735289 17409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.733679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:44853 (local address 127.0.0.1:55724)
0809 23:09:13.733881 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.733922 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.733981 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.734213 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.734218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.734224 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:13.734839 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:13.734839 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:13.734859 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:13.734887 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:13.735111 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:13.735118 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:13.735122 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.735195 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.735212 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":82,"threads_started":1}
I20260809 23:09:13.735303 17413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.733875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55724 (local address 127.8.141.195:44853)
0809 23:09:13.734064 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.734069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.734081 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:13.734112 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.734114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.734124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.734212 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.734963 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.734975 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.734975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.734980 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.735021 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.735051 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.735118 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.735203 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.735219 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":76,"threads_started":1}
I20260809 23:09:13.735791 17411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.733682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36651 (local address 127.0.0.1:41438)
0809 23:09:13.734122 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.734133 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.734194 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.734980 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.734981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.734983 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:13.735378 (+ 395us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:13.735379 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:13.735388 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:13.735405 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:13.735649 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:13.735650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:13.735651 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.735694 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.735708 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":136,"threads_started":1}
I20260809 23:09:13.735944 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.733884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34024 (local address 127.8.141.193:42997)
0809 23:09:13.734067 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.734071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.734564 (+ 493us) server_negotiation.cc:407] Connection header received
0809 23:09:13.734608 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.734610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.734618 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.734706 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.735621 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.735629 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.735629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.735634 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.735675 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.735699 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.735759 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.735835 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.735851 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260809 23:09:13.736088 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.733969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.8.141.194:36651)
0809 23:09:13.734137 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:13.734142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:13.734467 (+ 325us) server_negotiation.cc:407] Connection header received
0809 23:09:13.734504 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:13.734507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:13.734518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:13.734597 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:13.735507 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:13.735514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:13.735514 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:13.735518 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:13.735552 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:13.735576 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:13.735626 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:13.735961 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 23:09:13.735980 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
I20260809 23:09:13.736052 17294 tablet_service.cc:1467] Processing CreateTablet for tablet 37488bfb60ca407fb5c87ce6450e4197 (DEFAULT_TABLE table=test-table [id=7f4b00a7d9324134a867c5c5a46290d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:13.736349 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:13.733684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:42997 (local address 127.0.0.1:34024)
0809 23:09:13.734274 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:13.734283 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:13.734342 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:13.734708 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:13.734709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:13.734711 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:13.735518 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:13.735518 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:13.735526 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:13.735538 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:13.735760 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:13.735761 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:13.735762 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:09:13.735828 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:09:13.736284 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":127,"threads_started":1}
I20260809 23:09:13.736482 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37488bfb60ca407fb5c87ce6450e4197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:13.736862 17072 tablet_service.cc:1467] Processing CreateTablet for tablet 37488bfb60ca407fb5c87ce6450e4197 (DEFAULT_TABLE table=test-table [id=7f4b00a7d9324134a867c5c5a46290d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:13.737251 17072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37488bfb60ca407fb5c87ce6450e4197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:13.737327 16870 tablet_service.cc:1467] Processing CreateTablet for tablet 37488bfb60ca407fb5c87ce6450e4197 (DEFAULT_TABLE table=test-table [id=7f4b00a7d9324134a867c5c5a46290d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:13.737730 16870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37488bfb60ca407fb5c87ce6450e4197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:13.739887 17416 tablet_bootstrap.cc:492] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Bootstrap starting.
I20260809 23:09:13.740546 17416 tablet_bootstrap.cc:654] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:13.740913 17416 log.cc:824] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:13.740942 17417 tablet_bootstrap.cc:492] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Bootstrap starting.
I20260809 23:09:13.741451 17423 tablet_bootstrap.cc:492] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556: Bootstrap starting.
I20260809 23:09:13.741726 17417 tablet_bootstrap.cc:654] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:13.741856 17416 tablet_bootstrap.cc:492] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: No bootstrap required, opened a new log
I20260809 23:09:13.741936 17416 ts_tablet_manager.cc:1397] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:09:13.742091 17417 log.cc:824] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:13.742234 17423 tablet_bootstrap.cc:654] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:13.742575 17423 log.cc:824] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:13.743083 17417 tablet_bootstrap.cc:492] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: No bootstrap required, opened a new log
I20260809 23:09:13.743148 17417 ts_tablet_manager.cc:1397] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:09:13.743299 17423 tablet_bootstrap.cc:492] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556: No bootstrap required, opened a new log
I20260809 23:09:13.743342 17423 ts_tablet_manager.cc:1397] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:09:13.744916 17416 raft_consensus.cc:348] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:13.745095 17416 raft_consensus.cc:374] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:13.745132 17416 raft_consensus.cc:729] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bb5324291c648acbe44711281822808, State: Initialized, Role: FOLLOWER
I20260809 23:09:13.745072 17423 raft_consensus.cc:348] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:13.745182 17423 raft_consensus.cc:374] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:13.745203 17423 raft_consensus.cc:729] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81eea906a7004251abd96a9819f0d556, State: Initialized, Role: FOLLOWER
I20260809 23:09:13.745286 17416 consensus_queue.cc:260] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:13.745344 17423 consensus_queue.cc:260] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:13.745661 17423 ts_tablet_manager.cc:1428] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260809 23:09:13.745667 17416 ts_tablet_manager.cc:1428] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 23:09:13.745837 16983 heartbeater.cc:503] Master 127.8.141.254:43877 was elected leader, sending a full tablet report...
I20260809 23:09:13.745868 17395 heartbeater.cc:503] Master 127.8.141.254:43877 was elected leader, sending a full tablet report...
I20260809 23:09:13.746173 17417 raft_consensus.cc:348] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:13.746357 17417 raft_consensus.cc:374] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:13.746405 17417 raft_consensus.cc:729] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c55e716f83f74025b614182b3ea0a30b, State: Initialized, Role: FOLLOWER
I20260809 23:09:13.746531 17417 consensus_queue.cc:260] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:13.746873 17417 ts_tablet_manager.cc:1428] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260809 23:09:13.747009 17190 heartbeater.cc:503] Master 127.8.141.254:43877 was elected leader, sending a full tablet report...
W20260809 23:09:13.842820 17192 tablet.cc:2369] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:09:13.960760 16985 tablet.cc:2369] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:09:13.966642 17397 tablet.cc:2369] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:09:14.069442 17428 raft_consensus.cc:482] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:14.069623 17428 raft_consensus.cc:504] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:14.070103 17428 leader_election.cc:288] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81eea906a7004251abd96a9819f0d556 (127.8.141.193:42997), c55e716f83f74025b614182b3ea0a30b (127.8.141.194:36651)
I20260809 23:09:14.071735 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.070353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:42997 (local address 127.8.141.195:52927)
0809 23:09:14.070414 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:14.070427 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:14.070476 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:14.070683 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:14.070684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:14.070687 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:14.071297 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:14.071298 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:14.071306 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:14.071319 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:14.071562 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:14.071563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:14.071564 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:09:14.071647 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 23:09:14.071664 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 23:09:14.071748 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.070437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:52927 (local address 127.8.141.193:42997)
0809 23:09:14.070487 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.070489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.070500 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:14.070568 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.070568 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.070591 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.070649 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:14.071442 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:14.071444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:14.071445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:14.071450 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:14.071470 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:14.071475 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:14.071547 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:14.071668 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 23:09:14.071684 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:09:14.071938 17432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.070358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36651 (local address 127.8.141.195:57295)
0809 23:09:14.070651 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:14.070663 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:14.070745 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:14.071099 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:14.071100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:14.071120 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:14.071602 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:14.071602 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:14.071609 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:14.071619 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:14.071779 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:14.071780 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:14.071781 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 23:09:14.071829 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 23:09:14.071843 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":71,"thread_start_us":137,"threads_started":1}
I20260809 23:09:14.071973 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.070457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:57295 (local address 127.8.141.194:36651)
0809 23:09:14.070507 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.070510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.070717 (+ 207us) server_negotiation.cc:407] Connection header received
0809 23:09:14.070994 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.070995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.071002 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.071074 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:14.071704 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:14.071705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:14.071705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:14.071710 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:14.071725 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:14.071730 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:14.071784 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:14.071877 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 23:09:14.071895 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:09:14.072216 16897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37488bfb60ca407fb5c87ce6450e4197" candidate_uuid: "1bb5324291c648acbe44711281822808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81eea906a7004251abd96a9819f0d556" is_pre_election: true
I20260809 23:09:14.072407 16897 raft_consensus.cc:2454] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb5324291c648acbe44711281822808 in term 0.
I20260809 23:09:14.072440 17113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37488bfb60ca407fb5c87ce6450e4197" candidate_uuid: "1bb5324291c648acbe44711281822808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55e716f83f74025b614182b3ea0a30b" is_pre_election: true
I20260809 23:09:14.072608 17113 raft_consensus.cc:2454] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bb5324291c648acbe44711281822808 in term 0.
I20260809 23:09:14.072685 17224 leader_election.cc:302] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb5324291c648acbe44711281822808, 81eea906a7004251abd96a9819f0d556; no voters:
I20260809 23:09:14.072831 17428 raft_consensus.cc:2781] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:14.072933 17428 raft_consensus.cc:482] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:14.072952 17428 raft_consensus.cc:3037] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:14.073828 17428 raft_consensus.cc:504] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:14.073995 17428 leader_election.cc:288] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [CANDIDATE]: Term 1 election: Requested vote from peers 81eea906a7004251abd96a9819f0d556 (127.8.141.193:42997), c55e716f83f74025b614182b3ea0a30b (127.8.141.194:36651)
I20260809 23:09:14.074146 16897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37488bfb60ca407fb5c87ce6450e4197" candidate_uuid: "1bb5324291c648acbe44711281822808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81eea906a7004251abd96a9819f0d556"
I20260809 23:09:14.074203 17113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37488bfb60ca407fb5c87ce6450e4197" candidate_uuid: "1bb5324291c648acbe44711281822808" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c55e716f83f74025b614182b3ea0a30b"
I20260809 23:09:14.074240 16897 raft_consensus.cc:3037] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:14.074275 17113 raft_consensus.cc:3037] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:14.075259 16897 raft_consensus.cc:2454] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bb5324291c648acbe44711281822808 in term 1.
I20260809 23:09:14.075259 17113 raft_consensus.cc:2454] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bb5324291c648acbe44711281822808 in term 1.
I20260809 23:09:14.075508 17225 leader_election.cc:302] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bb5324291c648acbe44711281822808, c55e716f83f74025b614182b3ea0a30b; no voters:
I20260809 23:09:14.075632 17428 raft_consensus.cc:2781] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:14.075815 17428 raft_consensus.cc:686] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 1 LEADER]: Becoming Leader. State: Replica: 1bb5324291c648acbe44711281822808, State: Running, Role: LEADER
I20260809 23:09:14.075939 17428 consensus_queue.cc:237] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } }
I20260809 23:09:14.076882 16672 catalog_manager.cc:5495] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bb5324291c648acbe44711281822808 (127.8.141.195). New cstate: current_term: 1 leader_uuid: "1bb5324291c648acbe44711281822808" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bb5324291c648acbe44711281822808" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44853 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 } health_report { overall_health: UNKNOWN } } }
I20260809 23:09:14.092438 17413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.091189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55736 (local address 127.8.141.195:44853)
0809 23:09:14.091261 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.091263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.091270 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:09:14.091317 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.091318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.091333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.091403 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:14.092164 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:14.092167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:14.092167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:14.092178 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:14.092195 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:14.092198 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:14.092265 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:14.092322 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 23:09:14.092338 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 23:09:14.094626 17113 raft_consensus.cc:1261] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 1 FOLLOWER]: Refusing update from remote peer 1bb5324291c648acbe44711281822808: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:14.094626 16897 raft_consensus.cc:1261] T 37488bfb60ca407fb5c87ce6450e4197 P 81eea906a7004251abd96a9819f0d556 [term 1 FOLLOWER]: Refusing update from remote peer 1bb5324291c648acbe44711281822808: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:14.095019 17428 consensus_queue.cc:1035] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81eea906a7004251abd96a9819f0d556" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:14.095203 17433 consensus_queue.cc:1035] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c55e716f83f74025b614182b3ea0a30b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:14.098493 17439 mvcc.cc:205] Tried to move back new op lower bound from 7316754243968364544 to 7316754243895721984. Current Snapshot: MvccSnapshot[applied={T|T < 7316754243968364544}]
I20260809 23:09:14.099750 17445 mvcc.cc:205] Tried to move back new op lower bound from 7316754243968364544 to 7316754243895721984. Current Snapshot: MvccSnapshot[applied={T|T < 7316754243968364544}]
I20260809 23:09:14.101644 16666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:14.101771 16666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:14.102649 16666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cab6f5ef0c0940f1a9d64a0eb62c2dc3: Sending DeleteTablet for 3 replicas of tablet 37488bfb60ca407fb5c87ce6450e4197
I20260809 23:09:14.103016 17294 tablet_service.cc:1514] Processing DeleteTablet for tablet 37488bfb60ca407fb5c87ce6450e4197 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:14 UTC) from {username='slave'} at 127.0.0.1:55724
I20260809 23:09:14.103144 16870 tablet_service.cc:1514] Processing DeleteTablet for tablet 37488bfb60ca407fb5c87ce6450e4197 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:14 UTC) from {username='slave'} at 127.0.0.1:34024
I20260809 23:09:14.103195 17072 tablet_service.cc:1514] Processing DeleteTablet for tablet 37488bfb60ca407fb5c87ce6450e4197 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:14 UTC) from {username='slave'} at 127.0.0.1:41438
I20260809 23:09:14.103299 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 16763
I20260809 23:09:14.103503 17454 tablet_replica.cc:330] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: stopping tablet replica
I20260809 23:09:14.103647 17454 raft_consensus.cc:2229] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:14.103822 17454 raft_consensus.cc:2258] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:14.114632 17454 ts_tablet_manager.cc:1905] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:14.116864 17454 ts_tablet_manager.cc:1918] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:14.117063 17454 log.cc:1196] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/37488bfb60ca407fb5c87ce6450e4197
I20260809 23:09:14.117427 17454 ts_tablet_manager.cc:1939] T 37488bfb60ca407fb5c87ce6450e4197 P 1bb5324291c648acbe44711281822808: Deleting consensus metadata
I20260809 23:09:14.119477 17459 tablet_replica.cc:330] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: stopping tablet replica
I20260809 23:09:14.119649 17459 raft_consensus.cc:2229] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:14.119820 17459 raft_consensus.cc:2258] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:14.120328 16650 catalog_manager.cc:4841] TS 1bb5324291c648acbe44711281822808 (127.8.141.195:44853): tablet 37488bfb60ca407fb5c87ce6450e4197 (table test-table [id=7f4b00a7d9324134a867c5c5a46290d3]) successfully deleted
I20260809 23:09:14.120437 17459 ts_tablet_manager.cc:1905] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:14.122808 17459 ts_tablet_manager.cc:1918] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:14.122926 17459 log.cc:1196] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/37488bfb60ca407fb5c87ce6450e4197
I20260809 23:09:14.123342 17459 ts_tablet_manager.cc:1939] T 37488bfb60ca407fb5c87ce6450e4197 P c55e716f83f74025b614182b3ea0a30b: Deleting consensus metadata
I20260809 23:09:14.123932 16649 catalog_manager.cc:4841] TS c55e716f83f74025b614182b3ea0a30b (127.8.141.194:36651): tablet 37488bfb60ca407fb5c87ce6450e4197 (table test-table [id=7f4b00a7d9324134a867c5c5a46290d3]) successfully deleted
W20260809 23:09:14.130627 16648 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:42997 (error 108)
I20260809 23:09:14.130823 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 16987
I20260809 23:09:14.134331 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.134136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:42997 (local address 127.0.0.1:34030)
0809 23:09:14.134211 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:14.134276 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:42997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 23:09:14.134617 16648 catalog_manager.cc:4584] TS 81eea906a7004251abd96a9819f0d556 (127.8.141.193:42997): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37488bfb60ca407fb5c87ce6450e4197: Network error: Client connection negotiation failed: client connection to 127.8.141.193:42997: connect: Connection refused (error 111)
I20260809 23:09:14.158500 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 17196
I20260809 23:09:14.170985 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 16620
2026-08-09T23:09:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1024 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 23:09:14.192694 8759 mini_oidc.cc:152] Starting JWKS server
I20260809 23:09:14.194100 8759 webserver.cc:460] Webserver started at https://127.0.0.1:44247/ using document root <none> and password file <none>
I20260809 23:09:14.194187 8759 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 23:09:14.194393 8759 webserver.cc:460] Webserver started at http://127.0.0.1:37735/ using document root <none> and password file <none>
2026-08-09T23:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:14Z Disabled control of system clock
I20260809 23:09:14.204265 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33831
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:33301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:33831
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44247/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/testchainca.cert with env {}
W20260809 23:09:14.292181 17475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:14.292428 17475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:14.292452 17475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:14.293812 17475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:14.293964 17475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:14.294059 17475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:14.294085 17475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:14.294114 17475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:09:14.294142 17475 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:09:14.294167 17475 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44247/jwks.json
I20260809 23:09:14.296066 17475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:33831
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44247/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:14.296572 17475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:14.296921 17475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:14.302958 17484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:14.302929 17483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:14.303117 17486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:14.303438 17475 server_base.cc:1034] running on GCE node
I20260809 23:09:14.303885 17475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:14.304239 17475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:14.305420 17475 hybrid_clock.cc:648] HybridClock initialized: now 1786316954305392 us; error 43 us; skew 500 ppm
I20260809 23:09:14.306843 17475 webserver.cc:460] Webserver started at http://127.8.141.254:39377/ using document root <none> and password file <none>
I20260809 23:09:14.307121 17475 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:14.307192 17475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:14.307355 17475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:14.308452 17475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "ab280998610d434e8e4eda4f49423c40"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "bcb9ff5beb89935f31e1fc8b0578fe33"
server_key_iv: "41b8325643871b46e4efe29bb4cb8097"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.308835 17475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "ab280998610d434e8e4eda4f49423c40"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "bcb9ff5beb89935f31e1fc8b0578fe33"
server_key_iv: "41b8325643871b46e4efe29bb4cb8097"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.310504 17475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:14.311460 17494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:14.311712 17475 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:14.311832 17475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "ab280998610d434e8e4eda4f49423c40"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "bcb9ff5beb89935f31e1fc8b0578fe33"
server_key_iv: "41b8325643871b46e4efe29bb4cb8097"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.311928 17475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:14.321302 17475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:14.321766 17475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:14.321938 17475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:14.327730 17475 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:33831
I20260809 23:09:14.327796 17581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:33831 every 8 connection(s)
I20260809 23:09:14.328233 17475 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:14.329506 17583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:14.329674 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 17475
I20260809 23:09:14.329798 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:14.329999 8759 external_mini_cluster.cc:1428] Setting key 9693d571c1a3b9751bcbd6a12f52d419
I20260809 23:09:14.333129 17583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40: Bootstrap starting.
I20260809 23:09:14.333935 17583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:14.334297 17583 log.cc:824] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:14.335093 17583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40: No bootstrap required, opened a new log
I20260809 23:09:14.336216 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.330598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51482 (local address 127.8.141.254:33831)
0809 23:09:14.330838 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.330846 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.330874 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:09:14.331375 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.331382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.331452 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.331581 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:14.331883 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.332866 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.334047 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.334536 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.334685 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.335457 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:14.335461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:14.335462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:14.335674 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:14.335719 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:14.335733 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:14.335803 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:14.335895 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 23:09:14.335979 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260809 23:09:14.337392 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:09:14.337550 17583 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab280998610d434e8e4eda4f49423c40" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33831 } }
I20260809 23:09:14.337699 17583 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:14.337723 17583 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab280998610d434e8e4eda4f49423c40, State: Initialized, Role: FOLLOWER
I20260809 23:09:14.337872 17583 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab280998610d434e8e4eda4f49423c40" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33831 } }
I20260809 23:09:14.337945 17583 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:14.337970 17583 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:14.338034 17583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:14.338794 17583 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab280998610d434e8e4eda4f49423c40" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33831 } }
I20260809 23:09:14.338891 17583 leader_election.cc:302] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab280998610d434e8e4eda4f49423c40; no voters:
I20260809 23:09:14.339082 17583 leader_election.cc:288] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:14.339188 17591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:14.339375 17591 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [term 1 LEADER]: Becoming Leader. State: Replica: ab280998610d434e8e4eda4f49423c40, State: Running, Role: LEADER
I20260809 23:09:14.339378 17583 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:14.339572 17591 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab280998610d434e8e4eda4f49423c40" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33831 } }
I20260809 23:09:14.339999 17593 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab280998610d434e8e4eda4f49423c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab280998610d434e8e4eda4f49423c40" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33831 } } }
I20260809 23:09:14.340090 17593 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:14.340255 17592 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab280998610d434e8e4eda4f49423c40. Latest consensus state: current_term: 1 leader_uuid: "ab280998610d434e8e4eda4f49423c40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab280998610d434e8e4eda4f49423c40" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33831 } } }
I20260809 23:09:14.340322 17592 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:14.340304 17606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:14.340946 17606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:14.342523 17606 catalog_manager.cc:1347] Generated new cluster ID: aca505b3343343a69b28771ba37927e2
I20260809 23:09:14.342587 17606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:14.360409 17606 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:14.361161 17606 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:14.376223 17606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab280998610d434e8e4eda4f49423c40: Generated new TSK 0
I20260809 23:09:14.376479 17606 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 23:09:14.405239 17626 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 23:09:14.405437 17626 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 23:09:14.405790 17588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.389616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51492 (local address 127.8.141.254:33831)
0809 23:09:14.389702 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.389704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.389848 (+ 144us) server_negotiation.cc:407] Connection header received
0809 23:09:14.389971 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.389972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.389984 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.390046 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:09:14.390236 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.390883 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.392056 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.392260 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.392324 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.392586 (+ 262us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:09:14.405387 (+ 12801us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 23:09:14.405709 (+ 322us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51492: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 23:09:14.405869 17588 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51492: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 23:09:14.406708 17625 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:14.389550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33831 (local address 127.0.0.1:51492)
0809 23:09:14.389766 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:14.389779 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:14.389866 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:14.390076 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:14.390077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:14.390079 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:09:14.390158 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:14.390161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.391002 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.391003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:14.391961 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:14.391964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.392345 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.392346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:14.392416 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.392442 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:09:14.405756 (+ 13314us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 23:09:14.406170 (+ 414us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.141.254:33831: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":124,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.141.254:33831: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44247/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 23:09:14.407471 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 17475
2026-08-09T23:09:14Z chronyd exiting
I20260809 23:09:14.461747 8759 test_util.cc:175] -----------------------------------------------
I20260809 23:09:14.461880 8759 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786316926160006-8759-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (270 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 23:09:14.463264 8759 mini_oidc.cc:152] Starting JWKS server
I20260809 23:09:14.464098 8759 webserver.cc:460] Webserver started at https://127.0.0.1:43725/ using document root <none> and password file <none>
I20260809 23:09:14.464144 8759 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 23:09:14.464356 8759 webserver.cc:460] Webserver started at http://127.0.0.1:36539/ using document root <none> and password file <none>
2026-08-09T23:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:14Z Disabled control of system clock
I20260809 23:09:14.474640 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33901
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:33901
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43725/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/testchainca.cert with env {}
W20260809 23:09:14.562506 17633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:14.562740 17633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:14.562765 17633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:14.563941 17633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:14.563979 17633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:14.563992 17633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:14.564005 17633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:14.564018 17633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:09:14.564030 17633 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:09:14.564041 17633 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43725/jwks.json
I20260809 23:09:14.565994 17633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:33901
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43725/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:14.566236 17633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:14.566438 17633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:14.571817 17641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:14.572011 17633 server_base.cc:1034] running on GCE node
W20260809 23:09:14.571838 17646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:14.571821 17644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:14.572543 17633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:14.572877 17633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:14.574054 17633 hybrid_clock.cc:648] HybridClock initialized: now 1786316954574043 us; error 34 us; skew 500 ppm
I20260809 23:09:14.575449 17633 webserver.cc:460] Webserver started at http://127.8.141.254:38181/ using document root <none> and password file <none>
I20260809 23:09:14.575726 17633 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:14.575800 17633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:14.575960 17633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:14.577055 17633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "27a626187a9c48e0a45ea1013d036eba"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "bb94910a234c1bd9ec8ed44a18a2b383"
server_key_iv: "358c0c2c31d700ffc26147de83b13bd0"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.577502 17633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "27a626187a9c48e0a45ea1013d036eba"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "bb94910a234c1bd9ec8ed44a18a2b383"
server_key_iv: "358c0c2c31d700ffc26147de83b13bd0"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.579066 17633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:14.579985 17652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:14.580252 17633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:14.580368 17633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "27a626187a9c48e0a45ea1013d036eba"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "bb94910a234c1bd9ec8ed44a18a2b383"
server_key_iv: "358c0c2c31d700ffc26147de83b13bd0"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.580453 17633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:14.590940 17633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:14.591291 17633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:14.591449 17633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:14.595758 17633 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:33901
I20260809 23:09:14.595826 17741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:33901 every 8 connection(s)
I20260809 23:09:14.596205 17633 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:14.597292 17742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:14.599784 17742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba: Bootstrap starting.
I20260809 23:09:14.600203 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 17633
I20260809 23:09:14.600327 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:14.600523 8759 external_mini_cluster.cc:1428] Setting key 91bebb20096631f3c6a4fe60328899a9
I20260809 23:09:14.600592 17742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:14.600914 17742 log.cc:824] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:14.601823 17742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba: No bootstrap required, opened a new log
I20260809 23:09:14.604441 17742 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a626187a9c48e0a45ea1013d036eba" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33901 } }
I20260809 23:09:14.604624 17742 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:14.604674 17742 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27a626187a9c48e0a45ea1013d036eba, State: Initialized, Role: FOLLOWER
I20260809 23:09:14.604815 17742 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a626187a9c48e0a45ea1013d036eba" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33901 } }
I20260809 23:09:14.605021 17742 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:14.605086 17742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:14.605144 17742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:14.605875 17742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a626187a9c48e0a45ea1013d036eba" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33901 } }
I20260809 23:09:14.606009 17742 leader_election.cc:302] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27a626187a9c48e0a45ea1013d036eba; no voters:
I20260809 23:09:14.606227 17742 leader_election.cc:288] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:14.606256 17752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:14.606377 17752 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [term 1 LEADER]: Becoming Leader. State: Replica: 27a626187a9c48e0a45ea1013d036eba, State: Running, Role: LEADER
I20260809 23:09:14.606465 17752 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a626187a9c48e0a45ea1013d036eba" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33901 } }
I20260809 23:09:14.606513 17742 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:14.606801 17754 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27a626187a9c48e0a45ea1013d036eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a626187a9c48e0a45ea1013d036eba" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33901 } } }
I20260809 23:09:14.606907 17754 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:14.607143 17758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27a626187a9c48e0a45ea1013d036eba. Latest consensus state: current_term: 1 leader_uuid: "27a626187a9c48e0a45ea1013d036eba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27a626187a9c48e0a45ea1013d036eba" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33901 } } }
I20260809 23:09:14.607159 17761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:14.607239 17758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:14.607582 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.601088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57364 (local address 127.8.141.254:33901)
0809 23:09:14.601273 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.601280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.601304 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:09:14.602040 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.602049 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.602129 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.602293 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:14.602549 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.604035 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.605188 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.605775 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.605979 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.606714 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:14.606718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:14.606718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:14.606977 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:14.607026 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:14.607042 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:14.607103 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:14.607328 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 23:09:14.607431 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":61,"threads_started":1}
I20260809 23:09:14.608141 17761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:14.608632 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:09:14.609925 17761 catalog_manager.cc:1347] Generated new cluster ID: 90b0a5d935ea4139b2beee7db9d41c1e
I20260809 23:09:14.609984 17761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:14.629391 17761 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:14.630139 17761 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:14.639407 17761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27a626187a9c48e0a45ea1013d036eba: Generated new TSK 0
I20260809 23:09:14.639636 17761 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 23:09:14.666251 17791 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 23:09:14.666371 17791 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 23:09:14.667019 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.651237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57368 (local address 127.8.141.254:33901)
0809 23:09:14.651302 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.651305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.651348 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:09:14.651427 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.651429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.651441 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.651524 (+ 83us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:09:14.651622 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.652437 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.653307 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.653651 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.653741 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.653938 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:09:14.666728 (+ 12790us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 23:09:14.666926 (+ 198us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57368: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 23:09:14.667086 17748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57368: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 23:09:14.667176 17789 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:14.651031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:33901 (local address 127.0.0.1:57368)
0809 23:09:14.651277 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:14.651295 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:14.651357 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:14.651526 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:14.651526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:14.651528 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:09:14.651576 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:14.651578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.652549 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.652550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:14.653179 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:14.653181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.653764 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.653765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:14.653819 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.653825 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:09:14.666889 (+ 13064us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 23:09:14.667026 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.141.254:33901: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":169,"threads_started":1}
I20260809 23:09:14.667714 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 17633
2026-08-09T23:09:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (258 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 23:09:14.721544 8759 mini_oidc.cc:152] Starting JWKS server
I20260809 23:09:14.722293 8759 webserver.cc:460] Webserver started at https://127.0.0.1:41885/ using document root <none> and password file <none>
I20260809 23:09:14.722340 8759 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 23:09:14.722514 8759 webserver.cc:460] Webserver started at http://127.0.0.1:33883/ using document root <none> and password file <none>
2026-08-09T23:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:14Z Disabled control of system clock
I20260809 23:09:14.732009 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41769
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:41769
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41885/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 23:09:14.821792 17800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:14.822031 17800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:14.822055 17800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:14.823374 17800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:14.823433 17800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:14.823446 17800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:14.823458 17800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:14.823470 17800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:09:14.823483 17800 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:09:14.823494 17800 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41885/jwks.json
I20260809 23:09:14.825122 17800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:41769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41885/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:14.825450 17800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:14.825811 17800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:14.831214 17809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:14.831276 17814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:14.831276 17812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:14.831789 17800 server_base.cc:1034] running on GCE node
I20260809 23:09:14.832103 17800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:14.832456 17800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:14.833611 17800 hybrid_clock.cc:648] HybridClock initialized: now 1786316954833585 us; error 38 us; skew 500 ppm
I20260809 23:09:14.835063 17800 webserver.cc:460] Webserver started at http://127.8.141.254:42163/ using document root <none> and password file <none>
I20260809 23:09:14.835372 17800 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:14.835425 17800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:14.835577 17800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:14.836858 17800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "8d865eabd6dc42e094eb8d3107d75159"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "1dc5fa8a273c7cb22bc528e64fa4b552"
server_key_iv: "40859a424d1821de9fcba8d8b5d44682"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.837387 17800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "8d865eabd6dc42e094eb8d3107d75159"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "1dc5fa8a273c7cb22bc528e64fa4b552"
server_key_iv: "40859a424d1821de9fcba8d8b5d44682"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.838991 17800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:14.839968 17823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:14.840214 17800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:14.840328 17800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "8d865eabd6dc42e094eb8d3107d75159"
format_stamp: "Formatted at 2026-08-09 23:09:14 on dist-test-slave-j2vl"
server_key: "1dc5fa8a273c7cb22bc528e64fa4b552"
server_key_iv: "40859a424d1821de9fcba8d8b5d44682"
server_key_version: "encryptionkey@0"
I20260809 23:09:14.840396 17800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:14.855898 17800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:14.856246 17800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:14.856366 17800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:14.861047 17800 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:41769
I20260809 23:09:14.861088 17897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:41769 every 8 connection(s)
I20260809 23:09:14.861573 17800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:14.862421 17899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:14.865566 17899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159: Bootstrap starting.
I20260809 23:09:14.866240 17899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:14.866566 17899 log.cc:824] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:14.867466 17899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159: No bootstrap required, opened a new log
I20260809 23:09:14.867626 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 17800
I20260809 23:09:14.867785 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:14.867992 8759 external_mini_cluster.cc:1428] Setting key 37efd0a00d16569801ef02cc658e9f78
I20260809 23:09:14.870139 17899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d865eabd6dc42e094eb8d3107d75159" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41769 } }
I20260809 23:09:14.870348 17899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:14.870390 17899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d865eabd6dc42e094eb8d3107d75159, State: Initialized, Role: FOLLOWER
I20260809 23:09:14.870654 17899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d865eabd6dc42e094eb8d3107d75159" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41769 } }
I20260809 23:09:14.870807 17899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:14.870854 17899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:14.870919 17899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:14.871824 17899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d865eabd6dc42e094eb8d3107d75159" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41769 } }
I20260809 23:09:14.871946 17899 leader_election.cc:302] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d865eabd6dc42e094eb8d3107d75159; no voters:
I20260809 23:09:14.872119 17899 leader_election.cc:288] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:14.872187 17905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:14.872388 17905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [term 1 LEADER]: Becoming Leader. State: Replica: 8d865eabd6dc42e094eb8d3107d75159, State: Running, Role: LEADER
I20260809 23:09:14.872392 17899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:14.872509 17905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d865eabd6dc42e094eb8d3107d75159" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41769 } }
I20260809 23:09:14.872854 17906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d865eabd6dc42e094eb8d3107d75159. Latest consensus state: current_term: 1 leader_uuid: "8d865eabd6dc42e094eb8d3107d75159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d865eabd6dc42e094eb8d3107d75159" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41769 } } }
I20260809 23:09:14.872951 17906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:14.873180 17913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:14.873929 17913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:14.873962 17907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d865eabd6dc42e094eb8d3107d75159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d865eabd6dc42e094eb8d3107d75159" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41769 } } }
I20260809 23:09:14.874205 17907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:14.875777 17913 catalog_manager.cc:1347] Generated new cluster ID: 5d47ad9b857c455baa16d8a38f2b99ca
I20260809 23:09:14.875840 17913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:14.876567 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.868742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.8.141.254:41769)
0809 23:09:14.869014 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.869024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.869059 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:09:14.870073 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.870083 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.870185 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.870357 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:14.870704 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.872823 (+ 2119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.874032 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.874781 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.875330 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.875716 (+ 386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:14.875719 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:14.875720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:14.875936 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:14.875980 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:14.875995 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:14.876067 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:14.876178 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 23:09:14.876356 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":113,"threads_started":1}
I20260809 23:09:14.877512 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:09:14.895313 17913 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:14.896133 17913 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:14.900509 17913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d865eabd6dc42e094eb8d3107d75159: Generated new TSK 0
I20260809 23:09:14.900786 17913 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:09:14.914577 17903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:14.910096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50590 (local address 127.8.141.254:41769)
0809 23:09:14.910153 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:14.910155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:14.910496 (+ 341us) server_negotiation.cc:407] Connection header received
0809 23:09:14.910546 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:14.910547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:14.910559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:14.910637 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:09:14.910961 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.911544 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.912756 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.912986 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.913073 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.913304 (+ 231us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:09:14.914310 (+ 1006us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 23:09:14.914485 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 23:09:14.914656 17903 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50590: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 23:09:14.914664 17946 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:14.909939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41769 (local address 127.0.0.1:50590)
0809 23:09:14.910151 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:14.910168 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:14.910251 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:14.910664 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:14.910665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:14.910668 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:09:14.910740 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:14.910743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.911644 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.911646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:14.912627 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:14.912630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:14.913097 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:14.913098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:14.913161 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:14.913167 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:09:14.914438 (+ 1271us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 23:09:14.914547 (+ 109us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.141.254:41769: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41885/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":133,"threads_started":1}
I20260809 23:09:14.914878 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 17800
E20260809 23:09:14.936642 17947 webserver.cc:533] Webserver: sslize failed:
E20260809 23:09:14.936735 17947 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T23:09:14Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (245 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T23:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:14Z Disabled control of system clock
I20260809 23:09:14.976835 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:34131
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:41331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:34131
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/testchainca.cert with env {}
W20260809 23:09:15.067759 17953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:15.068053 17953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:15.068099 17953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:15.069640 17953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:15.069743 17953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:15.069768 17953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:15.069796 17953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:15.069830 17953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 23:09:15.069890 17953 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 23:09:15.069907 17953 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 23:09:15.071776 17953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:34131
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:34131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:15.072157 17953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:15.072430 17953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:15.078274 17968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:15.078554 17964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:15.078402 17965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:15.078845 17953 server_base.cc:1034] running on GCE node
I20260809 23:09:15.079187 17953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:15.079589 17953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:15.080749 17953 hybrid_clock.cc:648] HybridClock initialized: now 1786316955080686 us; error 85 us; skew 500 ppm
I20260809 23:09:15.085687 17953 webserver.cc:460] Webserver started at http://127.8.141.254:46007/ using document root <none> and password file <none>
I20260809 23:09:15.086030 17953 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:15.086090 17953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:15.086233 17953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:15.087394 17953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "b78e44c006e1460486e62abf2fbea1b4"
format_stamp: "Formatted at 2026-08-09 23:09:15 on dist-test-slave-j2vl"
server_key: "11a4fdd54d1f25214fbfd76ba701c44f"
server_key_iv: "fd9d15bb57a3cf8898aec1ae8c09d4f9"
server_key_version: "encryptionkey@0"
I20260809 23:09:15.088214 17953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "b78e44c006e1460486e62abf2fbea1b4"
format_stamp: "Formatted at 2026-08-09 23:09:15 on dist-test-slave-j2vl"
server_key: "11a4fdd54d1f25214fbfd76ba701c44f"
server_key_iv: "fd9d15bb57a3cf8898aec1ae8c09d4f9"
server_key_version: "encryptionkey@0"
I20260809 23:09:15.089756 17953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 23:09:15.090790 17976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:15.091089 17953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:15.091180 17953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "b78e44c006e1460486e62abf2fbea1b4"
format_stamp: "Formatted at 2026-08-09 23:09:15 on dist-test-slave-j2vl"
server_key: "11a4fdd54d1f25214fbfd76ba701c44f"
server_key_iv: "fd9d15bb57a3cf8898aec1ae8c09d4f9"
server_key_version: "encryptionkey@0"
I20260809 23:09:15.091284 17953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:15.113798 17953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:15.114177 17953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:15.114342 17953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:15.118685 17953 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:34131
I20260809 23:09:15.118723 18049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:34131 every 8 connection(s)
I20260809 23:09:15.119118 17953 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:15.120239 18050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:15.122745 18050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4: Bootstrap starting.
I20260809 23:09:15.123205 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 17953
I20260809 23:09:15.123330 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:15.123526 8759 external_mini_cluster.cc:1428] Setting key 3b8ed7ff67350f0b6595fd418d2bee65
I20260809 23:09:15.123857 18050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:15.124344 18050 log.cc:824] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:15.125423 18050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4: No bootstrap required, opened a new log
I20260809 23:09:15.127594 18050 raft_consensus.cc:348] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78e44c006e1460486e62abf2fbea1b4" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34131 } }
I20260809 23:09:15.127800 18050 raft_consensus.cc:374] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:15.127851 18050 raft_consensus.cc:729] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b78e44c006e1460486e62abf2fbea1b4, State: Initialized, Role: FOLLOWER
I20260809 23:09:15.127995 18050 consensus_queue.cc:260] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78e44c006e1460486e62abf2fbea1b4" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34131 } }
I20260809 23:09:15.128094 18050 raft_consensus.cc:388] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:15.128139 18050 raft_consensus.cc:482] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:15.128188 18050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:15.129037 18050 raft_consensus.cc:504] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78e44c006e1460486e62abf2fbea1b4" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34131 } }
I20260809 23:09:15.129150 18050 leader_election.cc:302] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b78e44c006e1460486e62abf2fbea1b4; no voters:
I20260809 23:09:15.129369 18050 leader_election.cc:288] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:15.129411 18057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:15.129521 18057 raft_consensus.cc:686] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [term 1 LEADER]: Becoming Leader. State: Replica: b78e44c006e1460486e62abf2fbea1b4, State: Running, Role: LEADER
I20260809 23:09:15.129608 18057 consensus_queue.cc:237] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78e44c006e1460486e62abf2fbea1b4" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34131 } }
I20260809 23:09:15.129671 18050 sys_catalog.cc:564] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:15.129930 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b78e44c006e1460486e62abf2fbea1b4. Latest consensus state: current_term: 1 leader_uuid: "b78e44c006e1460486e62abf2fbea1b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78e44c006e1460486e62abf2fbea1b4" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34131 } } }
I20260809 23:09:15.130002 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:15.130089 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b78e44c006e1460486e62abf2fbea1b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b78e44c006e1460486e62abf2fbea1b4" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 34131 } } }
I20260809 23:09:15.130177 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:15.130267 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:15.124077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.8.141.254:34131)
0809 23:09:15.124316 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:15.124323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:15.124354 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:09:15.124958 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:15.124973 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:15.125035 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:15.125155 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:15.125526 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:15.126473 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:15.127651 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:15.128312 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:15.128434 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:15.129359 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:15.129364 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:15.129365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:15.129620 (+ 255us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:15.129662 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:15.129678 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:15.129922 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:15.129959 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 23:09:15.130049 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260809 23:09:15.130944 18065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:15.131129 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:09:15.132207 18065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:15.134024 18065 catalog_manager.cc:1347] Generated new cluster ID: f7f64731fc9042b3acee4f4dc6ee9d72
I20260809 23:09:15.134100 18065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:15.144882 18065 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:15.145748 18065 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:15.151394 18065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b78e44c006e1460486e62abf2fbea1b4: Generated new TSK 0
I20260809 23:09:15.151631 18065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:09:15.577193 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:15.158732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60358 (local address 127.8.141.254:34131)
0809 23:09:15.158804 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:15.158806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:15.158913 (+ 107us) server_negotiation.cc:407] Connection header received
0809 23:09:15.158998 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:15.158999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:15.159013 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:15.159092 (+ 79us) server_negotiation.cc:226] Negotiated authn=JWT
0809 23:09:15.159264 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:15.160092 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:15.160961 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:15.161289 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:15.161392 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:15.161612 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 23:09:15.576948 (+415336us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 23:09:15.577126 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60358: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 23:09:15.577291 18053 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60358: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 23:09:15.577399 18100 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:15.158634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:34131 (local address 127.0.0.1:60358)
0809 23:09:15.158842 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:15.158853 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:15.158912 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:15.159118 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:15.159119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:15.159122 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0809 23:09:15.159182 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:15.159184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:15.160227 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:15.160228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:15.160861 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:15.160864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:15.161432 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:15.161434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:15.161490 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:15.161496 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 23:09:15.577114 (+415618us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 23:09:15.577246 (+ 132us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.141.254:34131: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":124,"threads_started":1}
I20260809 23:09:15.577966 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 17953
2026-08-09T23:09:15Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (636 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (98 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:15 dist-test-slave-j2vl krb5kdc[18117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:15 dist-test-slave-j2vl krb5kdc[18117](info): set up 2 sockets
Aug 09 23:09:15 dist-test-slave-j2vl krb5kdc[18117](info): commencing operation
krb5kdc: starting...
W20260809 23:09:17.099298 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:17.232839 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44015
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:44015
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:17.318432 18139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:17.318632 18139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:17.318655 18139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:17.319727 18139 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:17.319768 18139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:17.319782 18139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:17.319795 18139 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:17.319808 18139 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:17.321171 18139 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:44015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:17.321424 18139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:17.321614 18139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:09:17.326507 18139 server_base.cc:1034] running on GCE node
W20260809 23:09:17.326490 18149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:17.326493 18146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:17.326709 18147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:17.327000 18139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:17.327312 18139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:17.328496 18139 hybrid_clock.cc:648] HybridClock initialized: now 1786316957328471 us; error 41 us; skew 500 ppm
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.331115 18139 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:17.331588 18139 webserver.cc:460] Webserver started at http://127.8.141.254:36263/ using document root <none> and password file <none>
I20260809 23:09:17.331817 18139 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:17.331887 18139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:17.332041 18139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:17.333082 18139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "bae9fe9fae3b431694de52dc8112b54e"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "102517665c6c9de7fc2d41322927b4cd"
server_key_iv: "cf66fed8b890a3eef8f536201bcb5a41"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.333510 18139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "bae9fe9fae3b431694de52dc8112b54e"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "102517665c6c9de7fc2d41322927b4cd"
server_key_iv: "cf66fed8b890a3eef8f536201bcb5a41"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.334872 18139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:17.335600 18160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.335800 18139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.335904 18139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "bae9fe9fae3b431694de52dc8112b54e"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "102517665c6c9de7fc2d41322927b4cd"
server_key_iv: "cf66fed8b890a3eef8f536201bcb5a41"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.335991 18139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:17.343151 18139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:17.345086 18139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:17.345242 18139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:17.349676 18139 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:44015
I20260809 23:09:17.349867 18240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:44015 every 8 connection(s)
I20260809 23:09:17.350100 18139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:17.351044 18242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:17.353484 18242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e: Bootstrap starting.
I20260809 23:09:17.354100 18242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:17.354398 18242 log.cc:824] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:17.355182 18242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e: No bootstrap required, opened a new log
I20260809 23:09:17.356845 18242 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae9fe9fae3b431694de52dc8112b54e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44015 } }
I20260809 23:09:17.356990 18242 raft_consensus.cc:374] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:17.357013 18242 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae9fe9fae3b431694de52dc8112b54e, State: Initialized, Role: FOLLOWER
I20260809 23:09:17.357206 18242 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae9fe9fae3b431694de52dc8112b54e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44015 } }
I20260809 23:09:17.357295 18242 raft_consensus.cc:388] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:17.357369 18242 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:17.357419 18242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:17.357997 18242 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae9fe9fae3b431694de52dc8112b54e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44015 } }
I20260809 23:09:17.358101 18242 leader_election.cc:302] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bae9fe9fae3b431694de52dc8112b54e; no voters:
I20260809 23:09:17.358305 18242 leader_election.cc:288] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:17.358342 18248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:17.358502 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 18139
I20260809 23:09:17.358582 18248 raft_consensus.cc:686] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [term 1 LEADER]: Becoming Leader. State: Replica: bae9fe9fae3b431694de52dc8112b54e, State: Running, Role: LEADER
I20260809 23:09:17.358623 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:17.358619 18242 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:17.358678 18248 consensus_queue.cc:237] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae9fe9fae3b431694de52dc8112b54e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44015 } }
I20260809 23:09:17.358824 8759 external_mini_cluster.cc:1428] Setting key 3a0f3d4c7646b7cdd6076b18030d9ee7
I20260809 23:09:17.359071 18250 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bae9fe9fae3b431694de52dc8112b54e. Latest consensus state: current_term: 1 leader_uuid: "bae9fe9fae3b431694de52dc8112b54e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae9fe9fae3b431694de52dc8112b54e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44015 } } }
I20260809 23:09:17.359205 18249 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bae9fe9fae3b431694de52dc8112b54e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae9fe9fae3b431694de52dc8112b54e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44015 } } }
I20260809 23:09:17.359270 18249 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:17.359226 18250 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:17.359618 18260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:17.360342 18260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:17.361855 18260 catalog_manager.cc:1347] Generated new cluster ID: 4d2ba3f029944634b48941ea056937b5
I20260809 23:09:17.361918 18260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:17.370540 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.359524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42534 (local address 127.8.141.254:44015)
0809 23:09:17.359734 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.359745 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.361122 (+ 1377us) server_negotiation.cc:407] Connection header received
0809 23:09:17.361244 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.361252 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.361310 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.361490 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.362289 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.363216 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.363949 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.364394 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.364520 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.367130 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:17.367134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:17.367134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:17.367156 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:17.368852 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.369235 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.369238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.369239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.369391 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.369629 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.369630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.369631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.369884 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:17.370070 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:17.370168 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 23:09:17.370319 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260809 23:09:17.384582 18260 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:17.385317 18260 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:17.393857 18260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bae9fe9fae3b431694de52dc8112b54e: Generated new TSK 0
I20260809 23:09:17.394065 18260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:09:17.428462 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44015
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:17.538789 18278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:17.539028 18278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:17.539053 18278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:17.540297 18278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:17.540344 18278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:17.540371 18278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:17.541987 18278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:17.542204 18278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:17.542400 18278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:17.547838 18298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:17.547971 18278 server_base.cc:1034] running on GCE node
W20260809 23:09:17.547811 18286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:17.547868 18296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:17.548400 18278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:17.548664 18278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:17.549841 18278 hybrid_clock.cc:648] HybridClock initialized: now 1786316957549832 us; error 46 us; skew 500 ppm
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.552655 18278 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:09:17.553195 18278 webserver.cc:460] Webserver started at http://127.8.141.193:43505/ using document root <none> and password file <none>
I20260809 23:09:17.553510 18278 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:17.553586 18278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:17.553758 18278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:17.554908 18278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "ce92b24f848a4981826a0692307b88c3"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "70ae91bc428b92c3217585ad3dee2cfa"
server_key_iv: "1fefbadf9669c2ec1c4dba15e55f4ed8"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.555351 18278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "ce92b24f848a4981826a0692307b88c3"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "70ae91bc428b92c3217585ad3dee2cfa"
server_key_iv: "1fefbadf9669c2ec1c4dba15e55f4ed8"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.556846 18278 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:17.557816 18305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.558076 18278 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:09:17.558202 18278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "ce92b24f848a4981826a0692307b88c3"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "70ae91bc428b92c3217585ad3dee2cfa"
server_key_iv: "1fefbadf9669c2ec1c4dba15e55f4ed8"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.558285 18278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:17.568655 18278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:17.571094 18278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:17.571278 18278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:17.571547 18278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:17.572047 18278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:17.572132 18278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.572182 18278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:17.572216 18278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.579094 18278 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:44117
I20260809 23:09:17.579291 18487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:44117 every 8 connection(s)
I20260809 23:09:17.579558 18278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:17.585676 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 18278
I20260809 23:09:17.586086 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:17.586412 8759 external_mini_cluster.cc:1428] Setting key 5a84bb9668a1b8e90b5faf8717c406d0
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:17.594251 18490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.580423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44015 (local address 127.8.141.193:48373)
0809 23:09:17.580639 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:17.580689 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:17.581827 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:17.582251 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:17.582255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:17.582771 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:17.582997 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:17.583002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.584523 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.584524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:17.585477 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:17.585480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.586389 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.586391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:17.586567 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.587298 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:17.587310 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:17.589288 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:17.592249 (+ 2961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:17.592264 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:17.592267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:17.592735 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:17.593123 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:17.593125 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:17.593125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:17.593262 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:17.593670 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:17.593679 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:17.593825 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 23:09:17.593920 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:09:17.594066 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":128,"threads_started":1}
I20260809 23:09:17.594261 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.580481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:48373 (local address 127.8.141.254:44015)
0809 23:09:17.580596 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.580598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.580846 (+ 248us) server_negotiation.cc:407] Connection header received
0809 23:09:17.582022 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.582023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.582041 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.582109 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.583141 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.584339 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.585580 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.586204 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.586295 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.589440 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:17.589443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:17.589443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:17.589455 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:17.592015 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.592943 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.592944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.592945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.593019 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.593378 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.593380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.593380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.593565 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:17.593651 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:17.593997 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 23:09:17.594116 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:09:17.595016 18488 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44015
I20260809 23:09:17.595149 18488 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:17.595376 18488 heartbeater.cc:511] Master 127.8.141.254:44015 requested a full tablet report, sending...
I20260809 23:09:17.596205 18184 ts_manager.cc:194] Registered new tserver with Master: ce92b24f848a4981826a0692307b88c3 (127.8.141.193:44117)
I20260809 23:09:17.596875 18184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:48373
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:09:17.621511 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44015
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:17.711850 18494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:17.712069 18494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:17.712095 18494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:17.713299 18494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:17.713375 18494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:17.713405 18494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:17.715152 18494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:17.715422 18494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:17.715669 18494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:17.721524 18504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:17.721529 18502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:17.721525 18509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:17.721808 18494 server_base.cc:1034] running on GCE node
I20260809 23:09:17.722077 18494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:17.722304 18494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:17.723491 18494 hybrid_clock.cc:648] HybridClock initialized: now 1786316957723467 us; error 62 us; skew 500 ppm
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.726271 18494 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:09:17.726781 18494 webserver.cc:460] Webserver started at http://127.8.141.194:38711/ using document root <none> and password file <none>
I20260809 23:09:17.727032 18494 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:17.727103 18494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:17.727262 18494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:17.728415 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "8120a70a0c2b451d852c97e784b375d6"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "6cb00b8c903b8efcd1797b375824a673"
server_key_iv: "cab504bef4e15a361e7548b03f7edd23"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.728791 18494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "8120a70a0c2b451d852c97e784b375d6"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "6cb00b8c903b8efcd1797b375824a673"
server_key_iv: "cab504bef4e15a361e7548b03f7edd23"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.730362 18494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:09:17.731223 18521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.731436 18494 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:17.731545 18494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "8120a70a0c2b451d852c97e784b375d6"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "6cb00b8c903b8efcd1797b375824a673"
server_key_iv: "cab504bef4e15a361e7548b03f7edd23"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.731621 18494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:17.742244 18494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:17.744632 18494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:17.744810 18494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:17.745064 18494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:17.745496 18494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:17.745555 18494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.745599 18494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:17.745641 18494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.752012 18494 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:34761
I20260809 23:09:17.752143 18706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:34761 every 8 connection(s)
I20260809 23:09:17.752461 18494 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:17.758515 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 18494
I20260809 23:09:17.758610 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:17.758818 8759 external_mini_cluster.cc:1428] Setting key 469a21a6ba11a4d6fb53511d720e8c59
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:17.763063 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.753276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:34397 (local address 127.8.141.254:44015)
0809 23:09:17.753329 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.753331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.753595 (+ 264us) server_negotiation.cc:407] Connection header received
0809 23:09:17.754499 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.754500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.754512 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.754574 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.755209 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.755699 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.756799 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.757005 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.757070 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.759820 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:17.759822 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:17.759822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:17.759830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:17.761518 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.762006 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.762007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.762008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.762070 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.762343 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.762344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.762344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.762469 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:17.762544 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:17.762875 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 23:09:17.762970 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:09:17.763284 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.753224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44015 (local address 127.8.141.194:34397)
0809 23:09:17.753435 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:17.753479 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:17.754390 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:17.754626 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:17.754630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:17.754917 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:17.755121 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:17.755125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.755801 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.755802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:17.756645 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:17.756648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.757116 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.757117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:17.757235 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.757823 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:17.757832 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:17.759701 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:17.761596 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:17.761605 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:17.761608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:17.761929 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:17.762147 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:17.762148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:17.762148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:17.762271 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:17.762547 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:17.762555 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:17.762732 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 23:09:17.762847 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 23:09:17.763019 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
I20260809 23:09:17.763794 18707 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44015
I20260809 23:09:17.763896 18707 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:17.764106 18707 heartbeater.cc:511] Master 127.8.141.254:44015 requested a full tablet report, sending...
I20260809 23:09:17.764530 18184 ts_manager.cc:194] Registered new tserver with Master: 8120a70a0c2b451d852c97e784b375d6 (127.8.141.194:34761)
I20260809 23:09:17.765120 18184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:34397
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:09:17.789249 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44015
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:17.879113 18715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:17.879338 18715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:17.879365 18715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:17.880569 18715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:17.880614 18715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:17.880641 18715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:17.882302 18715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:17.882570 18715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:17.882819 18715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:09:17.888535 18715 server_base.cc:1034] running on GCE node
W20260809 23:09:17.888561 18722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:17.888561 18726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:17.888679 18724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:17.889025 18715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:17.889283 18715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:17.890445 18715 hybrid_clock.cc:648] HybridClock initialized: now 1786316957890432 us; error 32 us; skew 500 ppm
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.893204 18715 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:09:17.893780 18715 webserver.cc:460] Webserver started at http://127.8.141.195:42469/ using document root <none> and password file <none>
I20260809 23:09:17.894074 18715 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:17.894127 18715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:17.894223 18715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:17.895385 18715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "c3c755ce5fe94abe9a3251a8a771a389"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "77368a20b22ccbe3e159d45de15c6c4d"
server_key_iv: "293884215d29c541b5e53ebecda04fcc"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.895829 18715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "c3c755ce5fe94abe9a3251a8a771a389"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "77368a20b22ccbe3e159d45de15c6c4d"
server_key_iv: "293884215d29c541b5e53ebecda04fcc"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.897492 18715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:09:17.898406 18737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.898711 18715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:17.898804 18715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "c3c755ce5fe94abe9a3251a8a771a389"
format_stamp: "Formatted at 2026-08-09 23:09:17 on dist-test-slave-j2vl"
server_key: "77368a20b22ccbe3e159d45de15c6c4d"
server_key_iv: "293884215d29c541b5e53ebecda04fcc"
server_key_version: "encryptionkey@0"
I20260809 23:09:17.898916 18715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:17.910272 18715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:17.912599 18715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:17.912727 18715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:17.913038 18715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:17.913481 18715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:17.913543 18715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.913592 18715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:17.913619 18715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:17.920586 18715 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:35067
I20260809 23:09:17.920636 18926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:35067 every 8 connection(s)
I20260809 23:09:17.921000 18715 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:17.925550 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 18715
I20260809 23:09:17.925640 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:17.925851 8759 external_mini_cluster.cc:1428] Setting key 5d1ca00a9806e1c9cb73fe77cb764667
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:17.932401 18261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.921917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:51855 (local address 127.8.141.254:44015)
0809 23:09:17.922009 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.922012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.922211 (+ 199us) server_negotiation.cc:407] Connection header received
0809 23:09:17.923213 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.923214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.923227 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.923420 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.924027 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.924703 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.926228 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.926581 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.926688 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.929249 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:17.929252 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:17.929252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:17.929262 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:17.930892 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.931359 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.931360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.931361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.931430 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.931699 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.931700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.931701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.931832 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:17.931914 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:17.932201 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 23:09:17.932290 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:09:17.932631 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.921873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44015 (local address 127.8.141.195:51855)
0809 23:09:17.922071 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:17.922113 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:17.923090 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:17.923392 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:17.923395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:17.923675 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:17.923892 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:17.923896 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.924840 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.924841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:17.926119 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:17.926122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.926703 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.926704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:17.926911 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.927473 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:17.927483 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:17.929137 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:17.930986 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:17.930995 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:17.930998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:17.931248 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:17.931507 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:17.931508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:17.931508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:17.931631 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:17.931903 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:17.931912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:17.932065 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 23:09:17.932179 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:09:17.932365 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260809 23:09:17.933192 18927 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44015
I20260809 23:09:17.933296 18927 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:17.933517 18927 heartbeater.cc:511] Master 127.8.141.254:44015 requested a full tablet report, sending...
I20260809 23:09:17.933933 18184 ts_manager.cc:194] Registered new tserver with Master: c3c755ce5fe94abe9a3251a8a771a389 (127.8.141.195:35067)
I20260809 23:09:17.934283 18184 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:51855
I20260809 23:09:17.938529 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/test-admin.keytab.
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.960182 8759 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 23:09:17.962576 18939 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:17.961329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.8.141.193:44117)
0809 23:09:17.961683 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.961688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.961700 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:17.961749 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.961750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.961766 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.961885 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.962487 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33386: BlockingRecv error: recv got EOF from 127.0.0.1:33386 (error 108)
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":243,"threads_started":1}
W20260809 23:09:17.962815 8759 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc' of type FILE
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.964389 8759 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.965796 8759 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:09:17.975674 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.966945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33392 (local address 127.8.141.193:44117)
0809 23:09:17.967022 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.967024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.967033 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:17.967058 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.967059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.967070 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.967125 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.967624 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.968433 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.969232 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.969685 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.969749 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.972111 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:17.972122 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:17.972122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:17.972129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:17.973991 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.974386 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.974389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.974389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.974454 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.974744 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.974745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.974745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.975141 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:17.975278 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:17.975489 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 23:09:17.975580 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 23:09:17.978609 18939 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:17.977812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33400 (local address 127.8.141.193:44117)
0809 23:09:17.977959 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.977961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.978052 (+ 91us) server_negotiation.cc:407] Connection header received
0809 23:09:17.978134 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.978135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.978147 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.978206 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.978531 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33400: BlockingRecv error: recv got EOF from 127.0.0.1:33400 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260809 23:09:17.978752 8759 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc' of type FILE
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.980090 8759 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.981328 8759 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:09:17.990025 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.982094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.8.141.193:44117)
0809 23:09:17.982165 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.982167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.982179 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:17.982265 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.982266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.982287 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.982374 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.982828 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.983381 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.984158 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.984393 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.984497 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:17.987054 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:17.987056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:17.987056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:17.987065 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:17.988593 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.989020 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.989021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.989022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.989088 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:17.989415 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:17.989417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:17.989417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:17.989550 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:17.989701 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:17.989856 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 23:09:17.989950 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 23:09:17.992846 18939 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:17.991770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33416 (local address 127.8.141.193:44117)
0809 23:09:17.991855 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.991857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.992108 (+ 251us) server_negotiation.cc:407] Connection header received
0809 23:09:17.992173 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.992175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.992185 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.992238 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.992766 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33416: BlockingRecv error: recv got EOF from 127.0.0.1:33416 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 23:09:17.993068 8759 init.cc:244] Error while opening credential cache '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786316926160006-8759-0/krb5kdc/krb5cc)
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.994246 8759 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:17.995174 8759 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 23:09:17 dist-test-slave-j2vl krb5kdc[18117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:09:18.003230 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:17.995742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33424 (local address 127.8.141.193:44117)
0809 23:09:17.995817 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:17.995819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:17.995833 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:09:17.995933 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:17.995934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:17.995944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:17.995995 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:17.996383 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.996891 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.997684 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:17.997892 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:17.997974 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:18.000175 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:18.000177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:18.000177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:18.000187 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:18.002011 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:18.002369 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:18.002371 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:18.002371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:18.002436 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:18.002643 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:18.002644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:18.002644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:18.002792 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:18.002893 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:18.003034 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 23:09:18.003119 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:09:18.004073 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 18278
I20260809 23:09:18.024083 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 18494
I20260809 23:09:18.029886 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 18715
I20260809 23:09:18.052291 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 18139
2026-08-09T23:09:18Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2286 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:18 dist-test-slave-j2vl krb5kdc[18960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:18 dist-test-slave-j2vl krb5kdc[18960](info): set up 2 sockets
Aug 09 23:09:18 dist-test-slave-j2vl krb5kdc[18960](info): commencing operation
krb5kdc: starting...
W20260809 23:09:19.394580 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:19 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab.
Entry for principal oryx/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab.
I20260809 23:09:19.546108 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:37657
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab
--principal=oryx/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:37657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:19.635046 18980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:19.635308 18980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:19.635336 18980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:19.636603 18980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:19.636643 18980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:19.636657 18980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:19.636669 18980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:19.636682 18980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:19.638334 18980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:37657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab
--principal=oryx/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:37657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:19.638675 18980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:19.638902 18980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:19.644218 18991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:19.644236 18995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:19.644271 18980 server_base.cc:1034] running on GCE node
W20260809 23:09:19.644215 18990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:19.644887 18980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:19.645229 18980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:19.646425 18980 hybrid_clock.cc:648] HybridClock initialized: now 1786316959646408 us; error 51 us; skew 500 ppm
Aug 09 23:09:19 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:19.649147 18980 init.cc:377] Logged in from keytab as oryx/127.8.141.254@KRBTEST.COM (short username oryx)
I20260809 23:09:19.649683 18980 webserver.cc:460] Webserver started at http://127.8.141.254:44345/ using document root <none> and password file <none>
I20260809 23:09:19.649951 18980 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:19.650028 18980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:19.650184 18980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:19.651295 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "6bb0bc160f904210b3c30373b00b6cd2"
format_stamp: "Formatted at 2026-08-09 23:09:19 on dist-test-slave-j2vl"
server_key: "50f7f274a1a91880f3d2121adafa19de"
server_key_iv: "85c098ac9837b4c6e1be65444fd663fe"
server_key_version: "encryptionkey@0"
I20260809 23:09:19.651669 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "6bb0bc160f904210b3c30373b00b6cd2"
format_stamp: "Formatted at 2026-08-09 23:09:19 on dist-test-slave-j2vl"
server_key: "50f7f274a1a91880f3d2121adafa19de"
server_key_iv: "85c098ac9837b4c6e1be65444fd663fe"
server_key_version: "encryptionkey@0"
I20260809 23:09:19.653162 18980 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:19.654160 19005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:19.654372 18980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:19.654515 18980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "6bb0bc160f904210b3c30373b00b6cd2"
format_stamp: "Formatted at 2026-08-09 23:09:19 on dist-test-slave-j2vl"
server_key: "50f7f274a1a91880f3d2121adafa19de"
server_key_iv: "85c098ac9837b4c6e1be65444fd663fe"
server_key_version: "encryptionkey@0"
I20260809 23:09:19.654614 18980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:19.679482 18980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:19.681702 18980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:19.681876 18980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:19.686357 18980 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:37657
I20260809 23:09:19.686398 19089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:37657 every 8 connection(s)
I20260809 23:09:19.686802 18980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:19.687971 19090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:19.690348 19090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: Bootstrap starting.
I20260809 23:09:19.690994 19090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:19.691303 19090 log.cc:824] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:19.692106 19090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: No bootstrap required, opened a new log
I20260809 23:09:19.693262 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 18980
I20260809 23:09:19.693429 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:19.693646 8759 external_mini_cluster.cc:1428] Setting key 7addd85e8b8332aad9f83830f0d033f4
I20260809 23:09:19.694128 19090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } }
I20260809 23:09:19.694345 19090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:19.694415 19090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb0bc160f904210b3c30373b00b6cd2, State: Initialized, Role: FOLLOWER
I20260809 23:09:19.694556 19090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } }
I20260809 23:09:19.694634 19090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:19.694658 19090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:19.694753 19090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:19.695353 19090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } }
I20260809 23:09:19.695447 19090 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bb0bc160f904210b3c30373b00b6cd2; no voters:
I20260809 23:09:19.695710 19090 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:19.695740 19095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:19.695858 19095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 1 LEADER]: Becoming Leader. State: Replica: 6bb0bc160f904210b3c30373b00b6cd2, State: Running, Role: LEADER
I20260809 23:09:19.695942 19095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } }
I20260809 23:09:19.696013 19090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:19.696274 19095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } }
I20260809 23:09:19.696429 19095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:19.696617 19095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb0bc160f904210b3c30373b00b6cd2. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } }
I20260809 23:09:19.696761 19095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:19.697175 19101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:19.697808 19101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:19.699867 19101 catalog_manager.cc:1347] Generated new cluster ID: 24cacf5cdfbf4981b675e1041fd70f32
I20260809 23:09:19.699929 19101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:09:19 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:19.706954 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:19.694256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.8.141.254:37657)
0809 23:09:19.694510 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:19.694520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:19.694555 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:09:19.694731 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:19.694739 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:19.694798 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:19.694967 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:19.696003 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:19.697510 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:19.698321 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:19.699013 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:19.699146 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:19.702058 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:19.702062 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:19.702062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:19.702076 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:19.704807 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:19.705224 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:19.705227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:19.705228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:19.705382 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:19.705582 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:19.705582 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:19.705583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:19.705905 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:19.706052 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:19.706520 (+ 468us) server_negotiation.cc:299] Negotiation successful
0809 23:09:19.706675 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":121,"threads_started":1}
I20260809 23:09:19.712270 19101 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:19.713074 19101 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:19.724076 19101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: Generated new TSK 0
I20260809 23:09:19.724332 19101 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab.
Entry for principal oryx/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab.
I20260809 23:09:19.757416 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37657
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab
--principal=oryx/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:19.847128 19128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:19.847406 19128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:19.847450 19128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:19.848738 19128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:19.848817 19128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:19.848867 19128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:19.850555 19128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab
--principal=oryx/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:19.850872 19128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:19.851136 19128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:19.857096 19138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:19.857070 19139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:19.857070 19141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:19.857316 19128 server_base.cc:1034] running on GCE node
I20260809 23:09:19.857625 19128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:19.857931 19128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:19.859136 19128 hybrid_clock.cc:648] HybridClock initialized: now 1786316959859122 us; error 32 us; skew 500 ppm
Aug 09 23:09:19 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:19.861881 19128 init.cc:377] Logged in from keytab as oryx/127.8.141.193@KRBTEST.COM (short username oryx)
I20260809 23:09:19.862370 19128 webserver.cc:460] Webserver started at http://127.8.141.193:42653/ using document root <none> and password file <none>
I20260809 23:09:19.862628 19128 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:19.862675 19128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:19.862810 19128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:19.863930 19128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "57bb9cb93f01487c80ff6bade86d713f"
format_stamp: "Formatted at 2026-08-09 23:09:19 on dist-test-slave-j2vl"
server_key: "f7638d4aa356816ee4cbb469dcf7751c"
server_key_iv: "171ee1979a06ca3c698f8ee250d829e6"
server_key_version: "encryptionkey@0"
I20260809 23:09:19.864303 19128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "57bb9cb93f01487c80ff6bade86d713f"
format_stamp: "Formatted at 2026-08-09 23:09:19 on dist-test-slave-j2vl"
server_key: "f7638d4aa356816ee4cbb469dcf7751c"
server_key_iv: "171ee1979a06ca3c698f8ee250d829e6"
server_key_version: "encryptionkey@0"
I20260809 23:09:19.865777 19128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:19.866580 19149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:19.866811 19128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:19.866889 19128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "57bb9cb93f01487c80ff6bade86d713f"
format_stamp: "Formatted at 2026-08-09 23:09:19 on dist-test-slave-j2vl"
server_key: "f7638d4aa356816ee4cbb469dcf7751c"
server_key_iv: "171ee1979a06ca3c698f8ee250d829e6"
server_key_version: "encryptionkey@0"
I20260809 23:09:19.866943 19128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:19.883544 19128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:19.885733 19128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:19.885856 19128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:19.886070 19128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:19.886408 19128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:19.886467 19128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:19.886514 19128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:19.886538 19128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:19.892541 19128 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:41449
I20260809 23:09:19.892624 19360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:41449 every 8 connection(s)
I20260809 23:09:19.892979 19128 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:19.893960 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 19128
I20260809 23:09:19.894073 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:19.894289 8759 external_mini_cluster.cc:1428] Setting key dd49a760897cab44cee19e43f6dd5f36
Aug 09 23:09:19 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.8.141.194@KRBTEST.COM; defaulting to no policy
I20260809 23:09:19.903786 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:19.893898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:55033 (local address 127.8.141.254:37657)
0809 23:09:19.893962 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:19.893965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:19.894149 (+ 184us) server_negotiation.cc:407] Connection header received
0809 23:09:19.894890 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:19.894892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:19.894901 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:19.895055 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:19.896185 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:19.896720 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:19.898046 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:19.898259 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:19.898344 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:19.900665 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:19.900667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:19.900667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:19.900678 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:19.902223 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:19.902674 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:19.902677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:19.902678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:19.902737 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:19.902998 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:19.902999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:19.903000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:19.903126 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:19.903209 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:19.903584 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 23:09:19.903688 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:09:19.904073 19364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:19.893822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:55033)
0809 23:09:19.894017 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:19.894068 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:19.894784 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:19.895159 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:19.895164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:19.895803 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:19.896077 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:19.896082 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:19.896843 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:19.896844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:19.897957 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:19.897960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:19.898354 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:19.898356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:19.898499 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:19.899077 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:19.899087 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:19.900554 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:19.902332 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:19.902339 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:19.902342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:19.902584 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:19.902809 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:19.902810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:19.902810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:19.902927 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:19.903199 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:19.903207 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:19.903363 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 23:09:19.903546 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 23:09:19.903745 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260809 23:09:19.904942 19361 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37657
I20260809 23:09:19.905086 19361 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:19.905308 19361 heartbeater.cc:511] Master 127.8.141.254:37657 requested a full tablet report, sending...
I20260809 23:09:19.906303 19037 ts_manager.cc:194] Registered new tserver with Master: 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193:41449)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.194@KRBTEST.COM" created.
I20260809 23:09:19.907223 19037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:55033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab.
Entry for principal oryx/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab.
I20260809 23:09:19.925973 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37657
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab
--principal=oryx/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:20.016582 19367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:20.016816 19367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:20.016844 19367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:20.018117 19367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:20.018173 19367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:20.018201 19367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:20.019798 19367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab
--principal=oryx/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:20.020102 19367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:20.020370 19367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:20.026391 19378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.026393 19375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.026393 19373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.026786 19367 server_base.cc:1034] running on GCE node
I20260809 23:09:20.027016 19367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:20.027290 19367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:20.028460 19367 hybrid_clock.cc:648] HybridClock initialized: now 1786316960028419 us; error 57 us; skew 500 ppm
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:20.031419 19367 init.cc:377] Logged in from keytab as oryx/127.8.141.194@KRBTEST.COM (short username oryx)
I20260809 23:09:20.031958 19367 webserver.cc:460] Webserver started at http://127.8.141.194:36663/ using document root <none> and password file <none>
I20260809 23:09:20.032219 19367 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.032292 19367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.032449 19367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:20.033635 19367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "6512e359cf2642e4bb79c7f5605228df"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "db8a6a4ade38e2a9572b5fe925c9c8af"
server_key_iv: "04695f8b3481ee6249490d8eb5986b29"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.034026 19367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "6512e359cf2642e4bb79c7f5605228df"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "db8a6a4ade38e2a9572b5fe925c9c8af"
server_key_iv: "04695f8b3481ee6249490d8eb5986b29"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.035527 19367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:20.036355 19389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.036538 19367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:20.036651 19367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "6512e359cf2642e4bb79c7f5605228df"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "db8a6a4ade38e2a9572b5fe925c9c8af"
server_key_iv: "04695f8b3481ee6249490d8eb5986b29"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.036731 19367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:20.048664 19367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:20.051299 19367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:20.051494 19367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:20.051875 19367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:20.052279 19367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:20.052356 19367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.052421 19367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:20.052451 19367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.059780 19367 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:36881
I20260809 23:09:20.059827 19583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:36881 every 8 connection(s)
I20260809 23:09:20.060190 19367 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:20.062605 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 19367
I20260809 23:09:20.062738 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:20.062943 8759 external_mini_cluster.cc:1428] Setting key f1a04060f412c8837d0175c30fe3e285
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:20.072706 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.061089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:55007 (local address 127.8.141.254:37657)
0809 23:09:20.061147 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.061149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.061384 (+ 235us) server_negotiation.cc:407] Connection header received
0809 23:09:20.062095 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.062096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.062108 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.062178 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.063463 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.064105 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.066247 (+ 2142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.066586 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.066676 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.069240 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.069242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.069242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.069255 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.071162 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.071644 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.071647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.071648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.071709 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.071989 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.071990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.071990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.072116 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.072194 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.072501 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.072609 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:09:20.072922 19587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.061038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:55007)
0809 23:09:20.061232 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.061277 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.061829 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.062271 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.062275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.063086 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.063352 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.063359 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.064254 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.064256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.066119 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.066123 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.066760 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.066761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.066879 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.067554 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.067563 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.069119 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.071253 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.071262 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.071265 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.071548 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.071789 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.071790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.071790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.071911 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.072195 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.072204 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.072362 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.072480 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.072659 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260809 23:09:20.073472 19585 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37657
I20260809 23:09:20.073577 19585 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:20.073801 19585 heartbeater.cc:511] Master 127.8.141.254:37657 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.8.141.195@KRBTEST.COM; defaulting to no policy
I20260809 23:09:20.074232 19037 ts_manager.cc:194] Registered new tserver with Master: 6512e359cf2642e4bb79c7f5605228df (127.8.141.194:36881)
I20260809 23:09:20.074637 19037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:55007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab.
Entry for principal oryx/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab.
I20260809 23:09:20.098076 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37657
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab
--principal=oryx/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:20.186251 19593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:20.186472 19593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:20.186496 19593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:20.187687 19593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:20.187736 19593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:20.187770 19593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:20.189287 19593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab
--principal=oryx/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:20.189576 19593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:20.189838 19593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:20.195343 19605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.195454 19608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.195585 19593 server_base.cc:1034] running on GCE node
W20260809 23:09:20.195364 19603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.195906 19593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:20.196192 19593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:20.197432 19593 hybrid_clock.cc:648] HybridClock initialized: now 1786316960197395 us; error 49 us; skew 500 ppm
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:20.203922 19593 init.cc:377] Logged in from keytab as oryx/127.8.141.195@KRBTEST.COM (short username oryx)
I20260809 23:09:20.204493 19593 webserver.cc:460] Webserver started at http://127.8.141.195:42395/ using document root <none> and password file <none>
I20260809 23:09:20.204754 19593 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.204825 19593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.204982 19593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:20.206153 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "b1e6a8599056413c1e4b3fbf75099b71"
server_key_iv: "fe21840156269917f641902e1c0e65b5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.206579 19593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "b1e6a8599056413c1e4b3fbf75099b71"
server_key_iv: "fe21840156269917f641902e1c0e65b5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.208048 19593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:09:20.208832 19622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.209026 19593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:20.209120 19593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "b1e6a8599056413c1e4b3fbf75099b71"
server_key_iv: "fe21840156269917f641902e1c0e65b5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.209194 19593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:20.230504 19593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:20.233166 19593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:20.233500 19593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:20.233825 19593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:20.234194 19593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:20.234230 19593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.234279 19593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:20.234306 19593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.241230 19593 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:44743
I20260809 23:09:20.241317 19833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:44743 every 8 connection(s)
I20260809 23:09:20.241668 19593 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:20.244668 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 19593
I20260809 23:09:20.244788 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:20.244974 8759 external_mini_cluster.cc:1428] Setting key 9bcc8273ba7c6b16346115955f23b15b
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:20.251704 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.242535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:57265 (local address 127.8.141.254:37657)
0809 23:09:20.242630 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.242632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.242726 (+ 94us) server_negotiation.cc:407] Connection header received
0809 23:09:20.243315 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.243316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.243328 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.243398 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.244478 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.245116 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.246394 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.246583 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.246640 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.249030 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.249032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.249032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.249040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.250367 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.250766 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.250766 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.250767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.250821 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.251053 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.251053 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.251053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.251161 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.251216 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.251566 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.251633 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 23:09:20.251880 19841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.242439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:57265)
0809 23:09:20.242606 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.242655 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.243167 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.243512 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.243516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.244139 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.244342 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.244346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.245214 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.245215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.246318 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.246321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.246693 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.246695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.246840 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.247460 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.247469 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.248939 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.250459 (+ 1520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.250466 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.250469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.250697 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.250882 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.250882 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.250883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.250995 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.251260 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.251271 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.251410 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.251513 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.251650 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260809 23:09:20.252483 19838 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37657
I20260809 23:09:20.252578 19838 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:20.252791 19838 heartbeater.cc:511] Master 127.8.141.254:37657 requested a full tablet report, sending...
I20260809 23:09:20.253244 19037 ts_manager.cc:194] Registered new tserver with Master: 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743)
I20260809 23:09:20.253676 19037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:57265
I20260809 23:09:20.256996 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.141.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.253.keytab.
Entry for principal oryx/127.8.141.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.253.keytab.
I20260809 23:09:20.289638 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:41403
--webserver_interface=127.8.141.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.253.keytab
--principal=oryx/127.8.141.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:37657,127.8.141.253:41403
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:20.376698 19847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:20.376963 19847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:20.377002 19847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:20.378273 19847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:20.378343 19847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:20.378371 19847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:20.378401 19847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:20.378432 19847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:20.379962 19847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:37657,127.8.141.253:41403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.253.keytab
--principal=oryx/127.8.141.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:41403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.141.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
W20260809 23:09:20.380131 19847 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.141.254:37657,127.8.141.253:41403'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 23:09:20.380283 19847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:20.380515 19847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:20.385645 19856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.385654 19858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.385692 19847 server_base.cc:1034] running on GCE node
W20260809 23:09:20.385710 19853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.386086 19847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:20.386408 19847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:20.387580 19847 hybrid_clock.cc:648] HybridClock initialized: now 1786316960387555 us; error 40 us; skew 500 ppm
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:20.390416 19847 init.cc:377] Logged in from keytab as oryx/127.8.141.253@KRBTEST.COM (short username oryx)
I20260809 23:09:20.390890 19847 webserver.cc:460] Webserver started at http://127.8.141.253:44151/ using document root <none> and password file <none>
I20260809 23:09:20.391147 19847 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.391194 19847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.391373 19847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:20.392468 19847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/instance:
uuid: "34efe63281b04edca6d53f0e00394a28"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "aaad09a5901f476f6467765b1c43a539"
server_key_iv: "737ea181a84ebf066394b3e86dc86ea5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.392853 19847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal/instance:
uuid: "34efe63281b04edca6d53f0e00394a28"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "aaad09a5901f476f6467765b1c43a539"
server_key_iv: "737ea181a84ebf066394b3e86dc86ea5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.394377 19847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:20.395282 19875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.395639 19847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:20.395768 19847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
uuid: "34efe63281b04edca6d53f0e00394a28"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "aaad09a5901f476f6467765b1c43a539"
server_key_iv: "737ea181a84ebf066394b3e86dc86ea5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.395849 19847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:20.445168 19847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:20.447649 19847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:20.447841 19847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:20.453184 19847 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.253:41403
I20260809 23:09:20.453256 19964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.253:41403 every 8 connection(s)
I20260809 23:09:20.453675 19847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
I20260809 23:09:20.454753 19965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:20.456334 19965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:09:20.456719 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 19847
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.253@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.141.253@KRBTEST.COM
I20260809 23:09:20.467705 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.457105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48782 (local address 127.8.141.254:37657)
0809 23:09:20.457179 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.457181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.457396 (+ 215us) server_negotiation.cc:407] Connection header received
0809 23:09:20.458061 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.458063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.458077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.458130 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.458784 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.459298 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.460218 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.460430 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.460508 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.463610 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.463612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.463612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.463621 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.465472 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.466189 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.466191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.466191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.466263 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.466597 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.466598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.466598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.466804 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.466856 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.467432 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.467581 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:09:20.468063 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.457066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:48782)
0809 23:09:20.457253 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.457296 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.457959 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.458147 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.458151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.458506 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.458682 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.458687 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.459402 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.459403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.460126 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.460128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.460516 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.460517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.460646 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.461162 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.461171 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.463498 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.465564 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.465572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.465575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.466054 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.466335 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.466336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.466336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.466508 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.466911 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.466917 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.467250 (+ 333us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.467366 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.467552 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260809 23:09:20.468258 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.457414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.8.141.253:41403)
0809 23:09:20.457572 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.457576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.457686 (+ 110us) server_negotiation.cc:407] Connection header received
0809 23:09:20.458006 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.458023 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.458051 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.458189 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.458937 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.459756 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.460489 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.460785 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.460868 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.464501 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.464510 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.464511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.464518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.466618 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.467013 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.467017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.467017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.467082 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.467305 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.467306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.467306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.467482 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.467644 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.468058 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.468146 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260809 23:09:20.468935 19965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.253@KRBTEST.COM for oryx/127.8.141.253@KRBTEST.COM
I20260809 23:09:20.475380 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.469289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.0.0.1:57604)
0809 23:09:20.469366 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.469382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.469429 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.469553 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.469553 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.469714 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.469757 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.469759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.470450 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.470451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.470921 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.470923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.471291 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.471292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.471348 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.471738 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.471742 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.472932 (+ 1190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.474501 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.474502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.474502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.474685 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.474865 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.474865 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.474866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.474919 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.475128 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.475128 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.475166 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.475217 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.475296 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 23:09:20.475464 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.469305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57604 (local address 127.8.141.253:41403)
0809 23:09:20.469362 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.469364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.469420 (+ 56us) server_negotiation.cc:407] Connection header received
0809 23:09:20.469483 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.469483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.469494 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.469550 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.469822 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.470361 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.471003 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.471219 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.471295 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.473042 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.473043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.473044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.473050 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.474412 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.474756 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.474757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.474757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.474807 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.474974 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.474974 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.474974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.475074 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.475123 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.475230 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.475339 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":17}
I20260809 23:09:20.477044 19965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Bootstrap starting.
I20260809 23:09:20.477938 19965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:20.478214 19965 log.cc:824] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:20.479220 19965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: No bootstrap required, opened a new log
I20260809 23:09:20.480950 19965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } }
I20260809 23:09:20.481109 19965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:20.481137 19965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34efe63281b04edca6d53f0e00394a28, State: Initialized, Role: FOLLOWER
I20260809 23:09:20.481297 19965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } }
I20260809 23:09:20.481710 19979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } } }
I20260809 23:09:20.481787 19965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:20.481817 19979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:09:20.483700 19997 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:09:20.483772 19997 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:09:20.485857 19847 master_runner.cc:428] Detected that this master 34efe63281b04edca6d53f0e00394a28 is joining an existing cluster
I20260809 23:09:20.485945 19847 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.141.253:41403
I20260809 23:09:20.496299 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.487242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.8.141.254:37657)
0809 23:09:20.487293 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.487295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.487921 (+ 626us) server_negotiation.cc:407] Connection header received
0809 23:09:20.487948 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.487949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.487964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.488037 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.489469 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.490243 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.491329 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.491550 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.491648 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.493173 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.493175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.493175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.493182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.494710 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.495323 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.495324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.495324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.495392 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.495642 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.495642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.495643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.495771 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.495847 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.496052 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.496141 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:09:20.496258 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.487276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57612 (local address 127.8.141.253:41403)
0809 23:09:20.487314 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.487316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.487884 (+ 568us) server_negotiation.cc:407] Connection header received
0809 23:09:20.487940 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.487940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.487950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.487979 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.488675 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.489457 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.490517 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.490707 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.490795 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.492150 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.492152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.492153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.492161 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.494002 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.495287 (+ 1285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.495288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.495289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.495371 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.495685 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.495686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.495686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.495831 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.495877 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.496072 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.496169 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":20}
I20260809 23:09:20.496572 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.487200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.0.0.1:57612)
0809 23:09:20.487791 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.487804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.487877 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.488052 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.488052 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.488416 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.488585 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.488587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.489545 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.489546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.490435 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.490438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.490808 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.490808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.490881 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.491401 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.491407 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.491847 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.494092 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.494093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.494094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.495192 (+ 1098us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.495489 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.495489 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.495490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.495600 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.495924 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.495925 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.495964 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.496104 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.496255 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":41,"thread_start_us":109,"threads_started":1}
I20260809 23:09:20.496603 20010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.487596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:48794)
0809 23:09:20.487791 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.487804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.487882 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.488051 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.488052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.488416 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.488585 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.488587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.490348 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.490350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.491191 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.491194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.491651 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.491652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.491720 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.492430 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.492436 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.492873 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.494933 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.494935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.494935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.495263 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.495483 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.495484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.495484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.495562 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.495850 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.495851 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.495896 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.495983 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.496398 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":40,"thread_start_us":88,"threads_started":1}
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.254@KRBTEST.COM for oryx/127.8.141.253@KRBTEST.COM
I20260809 23:09:20.508863 19976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.500000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57626 (local address 127.8.141.253:41403)
0809 23:09:20.500107 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.500109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.500208 (+ 99us) server_negotiation.cc:407] Connection header received
0809 23:09:20.500318 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.500318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.500328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.500381 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.500968 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.501539 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.502808 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.502996 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.503061 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.505685 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.505687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.505687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.505694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.507373 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.507926 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.507927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.507927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.507990 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.508241 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.508243 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.508243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.508374 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.508436 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.508682 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.508757 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 23:09:20.508901 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.499877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.0.0.1:57626)
0809 23:09:20.500068 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.500126 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.500222 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.500411 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.500418 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.500808 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.500878 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.500881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.501622 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.501623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.502691 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.502694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.503080 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.503082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.503196 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.503707 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.503713 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.505585 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.507474 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.507484 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.507488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.507808 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.508073 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.508074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.508074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.508154 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.508450 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.508459 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.508526 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.508631 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.508781 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260809 23:09:20.509833 19037 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.141.253:41403: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "34efe63281b04edca6d53f0e00394a28"
member_type: NON_VOTER
last_known_addr {
host: "127.8.141.253"
port: 41403
}
attrs {
promote: true
}
}
dest_uuid: "6bb0bc160f904210b3c30373b00b6cd2"
cas_config_opid_index: -1
I20260809 23:09:20.510177 19037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } }
I20260809 23:09:20.510761 20016 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 34efe63281b04edca6d53f0e00394a28 (127.8.141.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } }
I20260809 23:09:20.511085 19927 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:20.511868 20016 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.141.253:41403
I20260809 23:09:20.512163 19927 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Refusing update from remote peer 6bb0bc160f904210b3c30373b00b6cd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 23:09:20.512178 20015 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } }
I20260809 23:09:20.512243 19847 master.cc:537] Master@127.8.141.253:41403 shutting down...
I20260809 23:09:20.512290 20015 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:20.512518 20016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 23:09:20.514902 19011 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 23:09:20.515375 19011 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 -> Peer 34efe63281b04edca6d53f0e00394a28 (127.8.141.253:41403): Couldn't send request to peer 34efe63281b04edca6d53f0e00394a28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 23:09:20.515843 19847 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:20.515954 19847 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:20.516009 19847 tablet_replica.cc:330] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: stopping tablet replica
I20260809 23:09:20.528892 19847 master.cc:552] Master@127.8.141.253:41403 shutdown complete.
I20260809 23:09:20.528985 19847 master_runner.cc:305] Clearing existing system tablet
I20260809 23:09:20.529135 19847 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.529208 19847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.530210 19847 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:20.530900 20023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.531035 19847 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:20.531097 19847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:20.531126 19847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
uuid: "34efe63281b04edca6d53f0e00394a28"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "aaad09a5901f476f6467765b1c43a539"
server_key_iv: "737ea181a84ebf066394b3e86dc86ea5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.531978 19847 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:20.533499 19847 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 23:09:20.533562 19847 log.cc:1196] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 23:09:20.533774 19847 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Deleting consensus metadata
I20260809 23:09:20.533923 19847 master_runner.cc:315] Copying system tablet from 127.8.141.254:37657
I20260809 23:09:20.534540 19847 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: tablet copy: Beginning tablet copy session from remote peer at address 127.8.141.254:37657
I20260809 23:09:20.541328 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.535038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48798 (local address 127.8.141.254:37657)
0809 23:09:20.535096 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.535098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.535161 (+ 63us) server_negotiation.cc:407] Connection header received
0809 23:09:20.535236 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.535236 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.535246 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.535297 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.535742 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.536259 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.537133 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.537334 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.537459 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.538518 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.538520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.538520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.538528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.540139 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.540489 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.540490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.540491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.540554 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.540745 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.540745 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.540746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.540871 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.540947 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.541146 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.541227 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:20.541380 20035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.534931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:48798)
0809 23:09:20.535067 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.535082 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.535165 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.535323 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.535324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.535573 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.535653 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.535656 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.536458 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.536459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.537028 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.537030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.537458 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.537459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.537534 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.538063 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.538069 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.538409 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.540212 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.540213 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.540213 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.540410 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.540623 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.540624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.540624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.540691 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.540949 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.540950 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.540998 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.541107 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.541231 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":87,"threads_started":1}
I20260809 23:09:20.541798 19069 tablet_copy_service.cc:140] P 6bb0bc160f904210b3c30373b00b6cd2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 34efe63281b04edca6d53f0e00394a28 ({username='oryx', principal='oryx/127.8.141.253@KRBTEST.COM'} at 127.0.0.1:48798)
I20260809 23:09:20.541926 19069 tablet_copy_service.cc:161] P 6bb0bc160f904210b3c30373b00b6cd2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 34efe63281b04edca6d53f0e00394a28 at {username='oryx', principal='oryx/127.8.141.253@KRBTEST.COM'} at 127.0.0.1:48798: session id = 34efe63281b04edca6d53f0e00394a28-00000000000000000000000000000000
I20260809 23:09:20.542698 19069 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 23:09:20.543431 19847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 23:09:20.544519 19847 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: tablet copy: Starting download of 0 data blocks...
I20260809 23:09:20.544691 19847 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: tablet copy: Starting download of 1 WAL segments...
I20260809 23:09:20.545805 19847 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 23:09:20.546706 19069 tablet_copy_service.cc:342] P 6bb0bc160f904210b3c30373b00b6cd2: Request end of tablet copy session 34efe63281b04edca6d53f0e00394a28-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.141.253@KRBTEST.COM'} at 127.0.0.1:48798
I20260809 23:09:20.546779 19069 tablet_copy_service.cc:433] P 6bb0bc160f904210b3c30373b00b6cd2: ending tablet copy session 34efe63281b04edca6d53f0e00394a28-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 34efe63281b04edca6d53f0e00394a28
I20260809 23:09:20.549968 19847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:20.551412 20038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.551417 20042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.551633 19847 server_base.cc:1034] running on GCE node
W20260809 23:09:20.551417 20037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.551852 19847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:20.552013 19847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:20.553254 19847 hybrid_clock.cc:648] HybridClock initialized: now 1786316960553264 us; error 44 us; skew 500 ppm
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:20.554765 19847 init.cc:377] Logged in from keytab as oryx/127.8.141.253@KRBTEST.COM (short username oryx)
I20260809 23:09:20.555088 19847 webserver.cc:460] Webserver started at http://127.8.141.253:35439/ using document root <none> and password file <none>
I20260809 23:09:20.555220 19847 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.555269 19847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.555966 19847 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:09:20.556541 20049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.556741 19847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:09:20.556803 19847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
uuid: "34efe63281b04edca6d53f0e00394a28"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "aaad09a5901f476f6467765b1c43a539"
server_key_iv: "737ea181a84ebf066394b3e86dc86ea5"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.556867 19847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:20.583951 19847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:20.585987 19847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:20.590018 19847 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.253:41403
I20260809 23:09:20.590143 20133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.253:41403 every 8 connection(s)
I20260809 23:09:20.590494 19847 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
I20260809 23:09:20.591333 20134 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:09:20.591776 20134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Bootstrap starting.
I20260809 23:09:20.594249 20134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:09:20.594496 20134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: Bootstrap complete.
I20260809 23:09:20.594712 20134 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } }
I20260809 23:09:20.594820 20134 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 34efe63281b04edca6d53f0e00394a28, State: Initialized, Role: LEARNER
I20260809 23:09:20.594874 20134 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } }
I20260809 23:09:20.595093 20134 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:20.595085 20141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } }
I20260809 23:09:20.595171 20141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: LEARNER
I20260809 23:09:20.596714 20164 catalog_manager.cc:1259] Loaded cluster ID: 24cacf5cdfbf4981b675e1041fd70f32
I20260809 23:09:20.596800 20164 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: loading cluster ID for follower catalog manager: success
I20260809 23:09:20.597878 20164 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: acquiring CA information for follower catalog manager: success
I20260809 23:09:20.598111 20164 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:09:20.629405 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.621586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.8.141.253:41403)
0809 23:09:20.621773 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.621776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.621786 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:09:20.621816 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.621817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.621827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.621922 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.622593 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.623494 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.624265 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.624490 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.624573 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.626202 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.626203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.626204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.626213 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.627979 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.628470 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.628471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.628471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.628541 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.628758 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.628758 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.628759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.628890 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.629013 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.629111 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.629217 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for oryx/127.8.141.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.252.keytab.
Entry for principal oryx/127.8.141.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.252.keytab.
I20260809 23:09:20.673436 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:34189
--webserver_interface=127.8.141.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.252.keytab
--principal=oryx/127.8.141.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:20.772372 20169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:20.772652 20169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:20.772696 20169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:20.774150 20169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:20.774227 20169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:20.774256 20169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:20.774292 20169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:20.774335 20169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:20.776116 20169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.252.keytab
--principal=oryx/127.8.141.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:34189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.141.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:20.776438 20169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:20.776708 20169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:20.782672 20176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.782763 20177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.782660 20181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.782998 20169 server_base.cc:1034] running on GCE node
I20260809 23:09:20.783278 20169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:20.783636 20169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:20.784852 20169 hybrid_clock.cc:648] HybridClock initialized: now 1786316960784838 us; error 35 us; skew 500 ppm
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:20.788036 20169 init.cc:377] Logged in from keytab as oryx/127.8.141.252@KRBTEST.COM (short username oryx)
I20260809 23:09:20.788511 20169 webserver.cc:460] Webserver started at http://127.8.141.252:33491/ using document root <none> and password file <none>
I20260809 23:09:20.788720 20169 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.788765 20169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.788884 20169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:20.790140 20169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/instance:
uuid: "33ad56731b30420dad50f4e3af1555ea"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "cf51a34ccfcb41d152a27ff3649e0bbe"
server_key_iv: "15a0316400bfadf78ba8c529e81ad6ac"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.790535 20169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal/instance:
uuid: "33ad56731b30420dad50f4e3af1555ea"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "cf51a34ccfcb41d152a27ff3649e0bbe"
server_key_iv: "15a0316400bfadf78ba8c529e81ad6ac"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.792289 20169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:20.793184 20190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.793504 20169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:20.793613 20169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
uuid: "33ad56731b30420dad50f4e3af1555ea"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "cf51a34ccfcb41d152a27ff3649e0bbe"
server_key_iv: "15a0316400bfadf78ba8c529e81ad6ac"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.793701 20169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:20.802256 20169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:20.805511 20169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:20.805718 20169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:20.811405 20169 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.252:34189
I20260809 23:09:20.811439 20283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.252:34189 every 8 connection(s)
I20260809 23:09:20.811841 20169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
I20260809 23:09:20.813009 20284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:20.814838 20284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:09:20.820847 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 20169
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.252@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:20.826217 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.815693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48810 (local address 127.8.141.254:37657)
0809 23:09:20.815747 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.815750 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.815844 (+ 94us) server_negotiation.cc:407] Connection header received
0809 23:09:20.816430 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.816431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.816449 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.816520 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.817445 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.817959 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.819078 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.819311 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.819388 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.822610 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.822612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.822612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.822620 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.824549 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.825062 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.825063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.825063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.825137 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.825463 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.825465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.825465 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.825627 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.825713 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.826066 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.826143 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:09:20.826499 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.815539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:48810)
0809 23:09:20.815724 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.815772 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.816302 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.816646 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.816650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.817038 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.817262 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.817267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.818056 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.818057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.818974 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.818977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.819455 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.819456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.819585 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.820191 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.820201 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.822474 (+ 2273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.824636 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.824644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.824646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.824983 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.825211 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.825211 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.825212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.825320 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.825703 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.825711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.825885 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.826005 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.826187 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260809 23:09:20.827148 20284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.141.252@KRBTEST.COM
I20260809 23:09:20.831838 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.821718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52404 (local address 127.8.141.252:34189)
0809 23:09:20.821875 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.821881 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.821904 (+ 23us) server_negotiation.cc:407] Connection header received
0809 23:09:20.821924 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.821926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.821960 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.822069 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.822833 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.823951 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.824888 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.825418 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.825493 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.828178 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.828186 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.828187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.828200 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.829955 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.830469 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.830472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.830472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.830568 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.830981 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.830982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.830982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.831202 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.831411 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.831612 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.831711 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.252@KRBTEST.COM for oryx/127.8.141.253@KRBTEST.COM
I20260809 23:09:20.835713 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.827550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.8.141.253:41403)
0809 23:09:20.827626 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.827628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.827636 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:20.827665 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.827666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.827686 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.827756 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.828080 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.828812 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.829733 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.829987 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.830072 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.832703 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.832704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.832705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.832713 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.834435 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.834849 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.834850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.834851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.834922 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.835135 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.835136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.835137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.835268 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.835349 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.835514 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.835612 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 23:09:20.835713 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.827455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.0.0.1:57656)
0809 23:09:20.827510 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.827521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.827562 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.827759 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.827760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.827922 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.827989 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.827991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.829009 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.829010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.829641 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.829644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.830066 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.830067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.830155 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.830697 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.830701 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.832587 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.834564 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.834565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.834565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.834769 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.834991 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.834992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.834992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.835062 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.835346 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.835346 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.835396 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.835485 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.835607 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:09:20.836598 20284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.252@KRBTEST.COM for oryx/127.8.141.252@KRBTEST.COM
I20260809 23:09:20.844795 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.837028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.8.141.252:34189)
0809 23:09:20.837064 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.837066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.837074 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:20.837150 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.837151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.837160 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.837212 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.837616 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.838120 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.838931 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.839176 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.839263 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.841789 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.841790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.841790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.841798 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.843454 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.843967 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.843969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.843969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.844031 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.844243 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.844244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.844244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.844372 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.844453 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.844582 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.844719 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":17}
I20260809 23:09:20.844790 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.836971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.0.0.1:52414)
0809 23:09:20.837030 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.837044 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.837089 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.837217 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.837217 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.837461 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.837503 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.837504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.838240 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.838241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.838772 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.838774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.839297 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.839298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.839451 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.840086 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.840091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.841655 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.843567 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.843569 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.843569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.843856 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.844117 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.844117 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.844118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.844180 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.844442 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.844443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.844484 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.844556 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.844669 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 23:09:20.846680 20284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Bootstrap starting.
I20260809 23:09:20.847713 20284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:20.848053 20284 log.cc:824] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:20.849201 20284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: No bootstrap required, opened a new log
I20260809 23:09:20.850787 20284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } }
I20260809 23:09:20.850950 20284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:20.850971 20284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ad56731b30420dad50f4e3af1555ea, State: Initialized, Role: FOLLOWER
I20260809 23:09:20.851114 20284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } }
I20260809 23:09:20.851502 20284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:20.851509 20289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } } }
I20260809 23:09:20.851675 20289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:09:20.853798 20313 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:09:20.853890 20313 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:09:20.857460 20169 master_runner.cc:428] Detected that this master 33ad56731b30420dad50f4e3af1555ea is joining an existing cluster
I20260809 23:09:20.857538 20169 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.141.252:34189
I20260809 23:09:20.882014 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.859252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.8.141.252:34189)
0809 23:09:20.859299 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.859302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.859420 (+ 118us) server_negotiation.cc:407] Connection header received
0809 23:09:20.859511 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.859512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.859521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.859548 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.860089 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.860868 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862116 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.862447 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862503 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.878786 (+ 16283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.878787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.878788 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.878794 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.880671 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881181 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.881182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.881182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.881233 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881450 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.881450 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.881451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.881592 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.881634 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.881818 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.881904 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:09:20.882174 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.858998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.8.141.254:37657)
0809 23:09:20.859055 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.859058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.859532 (+ 474us) server_negotiation.cc:407] Connection header received
0809 23:09:20.859631 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.859632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.859645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.859678 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.860271 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.860991 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862093 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.862391 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862437 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.878576 (+ 16139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.878579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.878580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.878590 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.880587 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881184 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.881186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.881187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.881255 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881511 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.881512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.881512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.881668 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.881716 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.881955 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.882049 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:09:20.882349 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.858902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.0.0.1:52416)
0809 23:09:20.859360 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.859370 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.859445 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.859607 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.859608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.859888 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.860005 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.860008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.860970 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.860971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.861919 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.861922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.862629 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.862698 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.878025 (+ 15327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.878033 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.878659 (+ 626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.880781 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.880782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.880782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.881102 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.881297 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.881298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.881386 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.881675 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.881676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.881712 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.882153 (+ 441us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.882252 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":235,"threads_started":1}
I20260809 23:09:20.882781 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.859116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.8.141.253:41403)
0809 23:09:20.859158 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.859160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.859522 (+ 362us) server_negotiation.cc:407] Connection header received
0809 23:09:20.859631 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.859632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.859645 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.859678 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.861190 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.861979 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862994 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.863200 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.863276 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.878798 (+ 15522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.878801 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.878801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.878812 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.881481 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881886 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.881888 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.881888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.881950 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.882156 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.882157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.882157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.882306 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.882363 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.882598 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.882685 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:09:20.883347 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 19128
I20260809 23:09:20.883376 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.859116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:48818)
0809 23:09:20.859473 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.859484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.859556 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.859748 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.859749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.860111 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.860176 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.860178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.861086 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.861087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.861990 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.861993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.862530 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.862601 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.863373 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.863379 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.878408 (+ 15029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.880713 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.880716 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.880717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.881076 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.881337 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881361 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.881361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.881431 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.881782 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.881783 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.881831 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.883091 (+ 1260us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.883245 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":280,"threads_started":1}
I20260809 23:09:20.886817 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.859013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.0.0.1:57662)
0809 23:09:20.859448 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.859459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.859554 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.859750 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.859750 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.861018 (+ 1268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.861101 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.861103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.862096 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.862097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.862898 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.862901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.863274 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.863275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.863350 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.863807 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.863813 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.878670 (+ 14857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.881569 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.881570 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.881570 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.881797 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.882027 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.882027 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.882028 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.882085 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.882424 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.882425 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.882475 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.883119 (+ 644us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.883312 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":95,"thread_start_us":314,"threads_started":1}
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316959, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.254@KRBTEST.COM for oryx/127.8.141.252@KRBTEST.COM
I20260809 23:09:20.901427 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.888351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.0.0.1:52432)
0809 23:09:20.888514 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.888526 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.888582 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.888724 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.888724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.888929 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.888978 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.888981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.889882 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.889883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.890734 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.890741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.891426 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.891428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.891526 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.893196 (+ 1670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.893202 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.895009 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.899932 (+ 4923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.899933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.899934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.900203 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.900428 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.900429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.900429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.900638 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.900987 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.900988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.901037 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.901095 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.901240 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260809 23:09:20.902163 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.888432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52432 (local address 127.8.141.252:34189)
0809 23:09:20.888512 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.888514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.888561 (+ 47us) server_negotiation.cc:407] Connection header received
0809 23:09:20.888653 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.888654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.888665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.888721 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.889067 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.889774 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.890843 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.891259 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.891338 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.895136 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.895138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.895138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.895148 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.899695 (+ 4547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.900292 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.900294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.900294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.900358 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.900738 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.900739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.900740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.900902 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.900960 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.901824 (+ 864us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.902033 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 23:09:20.918749 19037 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.141.252:34189: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "33ad56731b30420dad50f4e3af1555ea"
member_type: NON_VOTER
last_known_addr {
host: "127.8.141.252"
port: 34189
}
attrs {
promote: true
}
}
dest_uuid: "6bb0bc160f904210b3c30373b00b6cd2"
cas_config_opid_index: 5
I20260809 23:09:20.919088 19037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } }
I20260809 23:09:20.919495 20016 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 33ad56731b30420dad50f4e3af1555ea (127.8.141.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.920521 20016 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.141.252:34189
I20260809 23:09:20.920486 20228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:20.920594 20016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.920668 20016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:20.921066 20169 master.cc:537] Master@127.8.141.252:34189 shutting down...
I20260809 23:09:20.922158 20228 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Refusing update from remote peer 6bb0bc160f904210b3c30373b00b6cd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 23:09:20.922758 20015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:20.928591 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.919643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.0.0.1:57672)
0809 23:09:20.923543 (+ 3900us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.923557 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.923613 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.924016 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.924018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.924021 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:20.924094 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.924098 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.925312 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.925313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.926996 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.927001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.928208 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.928209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.928298 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.928338 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.928402 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.928463 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3877}
I20260809 23:09:20.928974 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.919785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.8.141.253:41403)
0809 23:09:20.923497 (+ 3712us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.923501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.923677 (+ 176us) server_negotiation.cc:407] Connection header received
0809 23:09:20.923711 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.923712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.923746 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.923815 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:20.924218 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.925190 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.927129 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.927957 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.928044 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.928160 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.928810 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.928870 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3672}
I20260809 23:09:20.929242 20086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 LEARNER]: Refusing update from remote peer 6bb0bc160f904210b3c30373b00b6cd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 23:09:20.929533 20015 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 23:09:20.931094 20086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 33ad56731b30420dad50f4e3af1555ea (127.8.141.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.933171 20141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb0bc160f904210b3c30373b00b6cd2. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.933290 20141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: LEARNER
I20260809 23:09:20.933365 20141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: NON_VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: true } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.933447 20141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: LEARNER
W20260809 23:09:20.936244 19008 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 -> Peer 33ad56731b30420dad50f4e3af1555ea (127.8.141.252:34189): Couldn't send request to peer 33ad56731b30420dad50f4e3af1555ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 23:09:20.938851 20015 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: attempting to promote NON_VOTER 34efe63281b04edca6d53f0e00394a28 to VOTER
I20260809 23:09:20.939119 20015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } }
I20260809 23:09:20.940472 20086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 LEARNER]: Refusing update from remote peer 6bb0bc160f904210b3c30373b00b6cd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 23:09:20.940689 20016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 23:09:20.941246 19008 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 -> Peer 33ad56731b30420dad50f4e3af1555ea (127.8.141.252:34189): Couldn't send request to peer 33ad56731b30420dad50f4e3af1555ea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 23:09:20.943024 20016 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 34efe63281b04edca6d53f0e00394a28 (127.8.141.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.944403 20016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.944516 20016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:20.944604 20016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.944684 20016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:20.945050 20086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 34efe63281b04edca6d53f0e00394a28 (127.8.141.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.946400 20141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:20.946519 20141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:09:20.950996 20169 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:20.951177 20169 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:20.951264 20169 tablet_replica.cc:330] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: stopping tablet replica
I20260809 23:09:20.955025 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:41449
--local_ip_for_outbound_sockets=127.8.141.193
--tserver_master_addrs=127.8.141.254:37657
--webserver_port=42653
--webserver_interface=127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab
--principal=oryx/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:09:20.965023 20169 master.cc:552] Master@127.8.141.252:34189 shutdown complete.
I20260809 23:09:20.965116 20169 master_runner.cc:305] Clearing existing system tablet
I20260809 23:09:20.965277 20169 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.965351 20169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.966153 20169 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:09:20.966789 20336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.967005 20169 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:20.967142 20169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:20.967188 20169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
uuid: "33ad56731b30420dad50f4e3af1555ea"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "cf51a34ccfcb41d152a27ff3649e0bbe"
server_key_iv: "15a0316400bfadf78ba8c529e81ad6ac"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.967927 20169 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:20.969554 20169 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 23:09:20.969658 20169 log.cc:1196] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 23:09:20.969950 20169 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Deleting consensus metadata
I20260809 23:09:20.970125 20169 master_runner.cc:315] Copying system tablet from 127.8.141.254:37657
I20260809 23:09:20.970850 20169 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: tablet copy: Beginning tablet copy session from remote peer at address 127.8.141.254:37657
I20260809 23:09:20.978494 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.971310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.8.141.254:37657)
0809 23:09:20.971370 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:20.971372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:20.971513 (+ 141us) server_negotiation.cc:407] Connection header received
0809 23:09:20.971594 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:20.971595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:20.971609 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:20.971675 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:20.972047 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.972625 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.973730 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.973980 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.974078 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.975241 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:20.975243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:20.975243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:20.975252 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:20.977039 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.977574 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.977575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.977575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.977645 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:20.977875 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:20.977876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:20.977876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:20.978011 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:20.978095 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:20.978283 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 23:09:20.978383 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:09:20.978500 20351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:20.971275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:48830)
0809 23:09:20.971426 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:20.971438 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:20.971517 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:20.971683 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:20.971683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:20.971894 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:20.971954 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.971956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.972738 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.972740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.973614 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:20.973618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:20.974084 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:20.974085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:20.974166 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:20.974741 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:20.974747 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:20.975116 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:20.977161 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.977163 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.977164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.977478 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.977722 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:20.977723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:20.977724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:20.977802 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:20.978098 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:20.978099 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:20.978155 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 23:09:20.978250 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:09:20.978383 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260809 23:09:20.979183 19069 tablet_copy_service.cc:140] P 6bb0bc160f904210b3c30373b00b6cd2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 33ad56731b30420dad50f4e3af1555ea ({username='oryx', principal='oryx/127.8.141.252@KRBTEST.COM'} at 127.0.0.1:48830)
I20260809 23:09:20.979274 19069 tablet_copy_service.cc:161] P 6bb0bc160f904210b3c30373b00b6cd2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 33ad56731b30420dad50f4e3af1555ea at {username='oryx', principal='oryx/127.8.141.252@KRBTEST.COM'} at 127.0.0.1:48830: session id = 33ad56731b30420dad50f4e3af1555ea-00000000000000000000000000000000
I20260809 23:09:20.979797 19069 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 23:09:20.980387 20169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 23:09:20.981592 20169 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: tablet copy: Starting download of 0 data blocks...
I20260809 23:09:20.981793 20169 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: tablet copy: Starting download of 1 WAL segments...
I20260809 23:09:20.982719 20169 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 23:09:20.983693 19069 tablet_copy_service.cc:342] P 6bb0bc160f904210b3c30373b00b6cd2: Request end of tablet copy session 33ad56731b30420dad50f4e3af1555ea-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.141.252@KRBTEST.COM'} at 127.0.0.1:48830
I20260809 23:09:20.983788 19069 tablet_copy_service.cc:433] P 6bb0bc160f904210b3c30373b00b6cd2: ending tablet copy session 33ad56731b30420dad50f4e3af1555ea-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 33ad56731b30420dad50f4e3af1555ea
I20260809 23:09:20.986924 20169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:20.988758 20354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.988771 20356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:20.988948 20361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:20.989133 20169 server_base.cc:1034] running on GCE node
I20260809 23:09:20.989390 20169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:20.989578 20169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:20.990734 20169 hybrid_clock.cc:648] HybridClock initialized: now 1786316960990724 us; error 26 us; skew 500 ppm
Aug 09 23:09:20 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316960, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:20.992321 20169 init.cc:377] Logged in from keytab as oryx/127.8.141.252@KRBTEST.COM (short username oryx)
I20260809 23:09:20.992659 20169 webserver.cc:460] Webserver started at http://127.8.141.252:33235/ using document root <none> and password file <none>
I20260809 23:09:20.992784 20169 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:20.992822 20169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:20.993563 20169 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:20.994140 20371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:20.994298 20169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 23:09:20.994350 20169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
uuid: "33ad56731b30420dad50f4e3af1555ea"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "cf51a34ccfcb41d152a27ff3649e0bbe"
server_key_iv: "15a0316400bfadf78ba8c529e81ad6ac"
server_key_version: "encryptionkey@0"
I20260809 23:09:20.994419 20169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:21.008002 20169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:21.010125 20169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:21.013826 20169 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.252:34189
I20260809 23:09:21.014247 20169 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
I20260809 23:09:21.015130 20456 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:09:21.015588 20456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Bootstrap starting.
I20260809 23:09:21.017433 20455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.252:34189 every 8 connection(s)
I20260809 23:09:21.018553 20456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:09:21.018869 20456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Bootstrap complete.
I20260809 23:09:21.019117 20456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } }
I20260809 23:09:21.019217 20456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 33ad56731b30420dad50f4e3af1555ea, State: Initialized, Role: LEARNER
I20260809 23:09:21.019266 20456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } }
I20260809 23:09:21.019459 20456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:21.019461 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:21.019740 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: This master's current role is: LEARNER
I20260809 23:09:21.021168 20481 catalog_manager.cc:1259] Loaded cluster ID: 24cacf5cdfbf4981b675e1041fd70f32
I20260809 23:09:21.021276 20481 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: loading cluster ID for follower catalog manager: success
I20260809 23:09:21.022744 20481 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: acquiring CA information for follower catalog manager: success
I20260809 23:09:21.023074 20481 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:09:21.075671 19585 heartbeater.cc:503] Master 127.8.141.254:37657 was elected leader, sending a full tablet report...
W20260809 23:09:21.084686 20334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:21.084987 20334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:21.085026 20334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:21.087061 20334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:21.087141 20334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:21.087182 20334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:21.089754 20334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab
--principal=oryx/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:41449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=42653
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:21.090083 20334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:21.090454 20334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:21.097095 20490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:21.097077 20492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:21.097177 20334 server_base.cc:1034] running on GCE node
W20260809 23:09:21.097385 20488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:21.097635 20334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:21.097985 20334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:21.099184 20334 hybrid_clock.cc:648] HybridClock initialized: now 1786316961099171 us; error 29 us; skew 500 ppm
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:21.102690 20334 init.cc:377] Logged in from keytab as oryx/127.8.141.193@KRBTEST.COM (short username oryx)
I20260809 23:09:21.103229 20334 webserver.cc:460] Webserver started at http://127.8.141.193:42653/ using document root <none> and password file <none>
I20260809 23:09:21.103487 20334 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:21.103551 20334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:21.105170 20334 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:21.106065 20501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.106356 20334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:21.106462 20334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "57bb9cb93f01487c80ff6bade86d713f"
format_stamp: "Formatted at 2026-08-09 23:09:19 on dist-test-slave-j2vl"
server_key: "f7638d4aa356816ee4cbb469dcf7751c"
server_key_iv: "171ee1979a06ca3c698f8ee250d829e6"
server_key_version: "encryptionkey@0"
I20260809 23:09:21.106874 20334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:21.116060 20334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:21.118605 20334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:21.118826 20334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:21.119179 20334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:21.119580 20334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:21.119619 20334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.119645 20334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:21.119659 20334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.128244 20334 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:41449
I20260809 23:09:21.128358 20683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:41449 every 8 connection(s)
I20260809 23:09:21.128757 20334 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:21.132560 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 20334
I20260809 23:09:21.132751 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 19367
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.253@KRBTEST.COM
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:21.142009 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.129844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:41717 (local address 127.8.141.253:41403)
0809 23:09:21.129968 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.129971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.130187 (+ 216us) server_negotiation.cc:407] Connection header received
0809 23:09:21.130873 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.130875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.130888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.130925 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.132029 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.132536 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.134330 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.134772 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.134835 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.138160 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.138162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.138162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.138171 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.140136 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.140708 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.140710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.140711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.140775 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.141110 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.141112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.141112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.141287 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.141373 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.141799 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.141902 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 23:09:21.142060 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.129792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:47665 (local address 127.8.141.254:37657)
0809 23:09:21.129922 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.129924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.130084 (+ 160us) server_negotiation.cc:407] Connection header received
0809 23:09:21.130854 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.130856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.130872 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.130907 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.132049 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.132889 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.134298 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.134604 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.134647 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.138804 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.138806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.138806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.138815 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.140771 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.141191 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.141193 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.141193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.141247 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.141491 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.141491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.141492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.141623 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.141667 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.141890 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.141968 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 23:09:21.142675 20694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.129728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:47665)
0809 23:09:21.129923 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.129978 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.130707 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.131069 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.131074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.131656 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.131921 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.131928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.133002 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.133003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.134194 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.134197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.134754 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.134755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.134902 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.135550 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.135561 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.138378 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.140875 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.140876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.140876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.141105 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.141322 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.141323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.141324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.141416 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.141716 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.141717 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.141760 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.142212 (+ 452us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.142409 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":6,"thread_start_us":96,"threads_started":1}
I20260809 23:09:21.143257 20696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.129728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.8.141.193:41717)
0809 23:09:21.130116 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.130128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.130733 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.131368 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.131369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.131655 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.131920 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.131925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.132661 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.132662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.134201 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.134205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.134933 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.134934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.135024 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.135863 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.135868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.137999 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.140243 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.140254 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.140256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.140607 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.140856 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.140857 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.140857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.141021 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.141423 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.141431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.141613 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.142566 (+ 953us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.142663 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":210,"thread_start_us":64,"threads_started":1}
I20260809 23:09:21.143257 20687 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37657
I20260809 23:09:21.143603 20687 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.143847 20687 heartbeater.cc:511] Master 127.8.141.254:37657 requested a full tablet report, sending...
I20260809 23:09:21.144280 19037 ts_manager.cc:194] Re-registered known tserver with Master: 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193:41449)
I20260809 23:09:21.144690 19037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:47665
I20260809 23:09:21.151086 20688 heartbeater.cc:347] Connected to a master server at 127.8.141.253:41403
I20260809 23:09:21.151175 20688 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.151288 20688 heartbeater.cc:511] Master 127.8.141.253:41403 requested a full tablet report, sending...
I20260809 23:09:21.151803 20072 ts_manager.cc:194] Registered new tserver with Master: 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193:41449)
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.252@KRBTEST.COM
I20260809 23:09:21.156409 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:36881
--local_ip_for_outbound_sockets=127.8.141.194
--tserver_master_addrs=127.8.141.254:37657
--webserver_port=36663
--webserver_interface=127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab
--principal=oryx/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:09:21.157447 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.129995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:57581 (local address 127.8.141.252:34189)
0809 23:09:21.130128 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.130131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.130142 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:21.130859 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.130861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.130880 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.130960 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.132049 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.150338 (+ 18289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.151688 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.152089 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.152161 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.154264 (+ 2103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.154265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.154266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.154274 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.155906 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.156331 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.156332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.156332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.156409 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.156650 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.156652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.156652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.156819 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.156892 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.157156 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.157276 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
I20260809 23:09:21.157467 20697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.129728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.8.141.193:57581)
0809 23:09:21.130080 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.130091 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.130707 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.131069 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.131074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.131648 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.131921 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.131928 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.150505 (+ 18577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.150507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.151574 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.151579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.152195 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.152197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.152278 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.152844 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.152848 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.154156 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.155990 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.155991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.155992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.156242 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.156488 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.156489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.156489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.156566 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.156956 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.156959 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.157011 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.157106 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.157299 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":51,"thread_start_us":85,"threads_started":1}
I20260809 23:09:21.158782 20685 heartbeater.cc:347] Connected to a master server at 127.8.141.252:34189
I20260809 23:09:21.158846 20685 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.158919 20685 heartbeater.cc:511] Master 127.8.141.252:34189 requested a full tablet report, sending...
I20260809 23:09:21.159436 20392 ts_manager.cc:194] Registered new tserver with Master: 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193:41449)
W20260809 23:09:21.246770 20700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:21.246999 20700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:21.247025 20700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:21.248205 20700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:21.248248 20700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:21.248275 20700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:21.249864 20700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab
--principal=oryx/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:36881
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=36663
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:21.250077 20700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:21.250278 20700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:09:21.254801 19838 heartbeater.cc:503] Master 127.8.141.254:37657 was elected leader, sending a full tablet report...
W20260809 23:09:21.255859 20708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:21.255986 20711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:21.256045 20709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:21.256178 20700 server_base.cc:1034] running on GCE node
I20260809 23:09:21.256526 20700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:21.256819 20700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:21.258035 20700 hybrid_clock.cc:648] HybridClock initialized: now 1786316961257997 us; error 50 us; skew 500 ppm
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:21.261112 20700 init.cc:377] Logged in from keytab as oryx/127.8.141.194@KRBTEST.COM (short username oryx)
I20260809 23:09:21.261722 20700 webserver.cc:460] Webserver started at http://127.8.141.194:36663/ using document root <none> and password file <none>
I20260809 23:09:21.262082 20700 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:21.262207 20700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:21.263746 20700 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:21.264523 20726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.264751 20700 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:21.264873 20700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "6512e359cf2642e4bb79c7f5605228df"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "db8a6a4ade38e2a9572b5fe925c9c8af"
server_key_iv: "04695f8b3481ee6249490d8eb5986b29"
server_key_version: "encryptionkey@0"
I20260809 23:09:21.265283 20700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:21.312839 20700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:21.315011 20700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:21.315181 20700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:21.315454 20700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:21.315845 20700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:21.315903 20700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.315949 20700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:21.315976 20700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.322914 20700 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:36881
I20260809 23:09:21.322959 20901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:36881 every 8 connection(s)
I20260809 23:09:21.323343 20700 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:21.324677 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 20700
I20260809 23:09:21.324839 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 19593
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.252@KRBTEST.COM
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:21.340804 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.324361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:52243 (local address 127.8.141.252:34189)
0809 23:09:21.324429 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.324431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.324870 (+ 439us) server_negotiation.cc:407] Connection header received
0809 23:09:21.329826 (+ 4956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.329828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.329845 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.329938 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.331130 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.331618 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.333465 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.333753 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.333846 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.336735 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.336737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.336738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.336747 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.338676 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.339384 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.339386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.339386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.339457 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.339769 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.339770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.339770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.339946 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.339994 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.340525 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.340646 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:21.341804 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.324280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.8.141.194:52243)
0809 23:09:21.324712 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.324764 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.329737 (+ 4973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.329947 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.329948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.330174 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.330376 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.330381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.331719 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.331720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.332984 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.332988 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.333863 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.333864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.334040 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.334762 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.334773 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.336606 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.338780 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.338792 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.338795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.339282 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.339533 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.339534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.339534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.339694 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.340045 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.340053 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.340267 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.341209 (+ 942us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.341476 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":30,"thread_start_us":94,"threads_started":1}
I20260809 23:09:21.342639 20902 heartbeater.cc:347] Connected to a master server at 127.8.141.252:34189
I20260809 23:09:21.342823 20902 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.343084 20902 heartbeater.cc:511] Master 127.8.141.252:34189 requested a full tablet report, sending...
I20260809 23:09:21.343614 20392 ts_manager.cc:194] Registered new tserver with Master: 6512e359cf2642e4bb79c7f5605228df (127.8.141.194:36881)
I20260809 23:09:21.344882 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.324337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:58077 (local address 127.8.141.254:37657)
0809 23:09:21.324395 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.324397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.327613 (+ 3216us) server_negotiation.cc:407] Connection header received
0809 23:09:21.328627 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.328629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.328647 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.328684 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.330502 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.331031 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.333104 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.333390 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.333478 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.340774 (+ 7296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.340777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.340777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.340787 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.343581 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.343995 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.343996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.343996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.344061 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.344270 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.344271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.344272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.344458 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.344525 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.344704 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.344788 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 23:09:21.344906 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.324468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:58077)
0809 23:09:21.327526 (+ 3058us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.327539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.328467 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.328934 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.328940 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.330107 (+ 1167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.330378 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.330386 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.331163 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.331165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.332982 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.332986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.338126 (+ 5140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.338128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.338237 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.338732 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.338737 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.340659 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.343683 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.343683 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.343684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.343905 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.344134 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.344135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.344135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.344196 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.344515 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.344516 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.344562 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.344660 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.344780 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3035,"mutex_wait_us":34,"thread_start_us":114,"threads_started":1}
I20260809 23:09:21.345176 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:44743
--local_ip_for_outbound_sockets=127.8.141.195
--tserver_master_addrs=127.8.141.254:37657
--webserver_port=42395
--webserver_interface=127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab
--principal=oryx/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.253@KRBTEST.COM
I20260809 23:09:21.345494 20903 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37657
I20260809 23:09:21.345590 20903 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.345671 20903 heartbeater.cc:511] Master 127.8.141.254:37657 requested a full tablet report, sending...
I20260809 23:09:21.345919 19037 ts_manager.cc:194] Re-registered known tserver with Master: 6512e359cf2642e4bb79c7f5605228df (127.8.141.194:36881)
I20260809 23:09:21.346449 19037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:58077
I20260809 23:09:21.349992 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.339675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:34567 (local address 127.8.141.253:41403)
0809 23:09:21.339726 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.339729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.339737 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:21.339764 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.339765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.339778 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.339845 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.340264 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.341019 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.341837 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.342127 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.342377 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.345795 (+ 3418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.345797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.345797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.345806 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.348619 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.349109 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.349110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.349111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.349175 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.349439 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.349440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.349440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.349574 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.349629 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.349846 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.349922 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:09:21.350374 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.338716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.8.141.194:34567)
0809 23:09:21.339482 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.339496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.339559 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.339898 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.339899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.340086 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.340169 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.340171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.341118 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.341119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.341730 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.341732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.342445 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.342446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.342525 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.343405 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.343410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.345671 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.348705 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.348707 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.348707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.349007 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.349256 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.349257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.349257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.349337 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.349653 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.349654 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.349718 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.349800 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.349988 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":103,"threads_started":1}
I20260809 23:09:21.350880 20905 heartbeater.cc:347] Connected to a master server at 127.8.141.253:41403
I20260809 23:09:21.350962 20905 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.351076 20905 heartbeater.cc:511] Master 127.8.141.253:41403 requested a full tablet report, sending...
I20260809 23:09:21.351403 20072 ts_manager.cc:194] Registered new tserver with Master: 6512e359cf2642e4bb79c7f5605228df (127.8.141.194:36881)
W20260809 23:09:21.446178 20913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:21.446424 20913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:21.446467 20913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:21.447722 20913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:21.447793 20913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:21.447839 20913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:21.449477 20913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab
--principal=oryx/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:44743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=42395
--tserver_master_addrs=127.8.141.254:37657,127.8.141.253:41403,127.8.141.252:34189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:21.449744 20913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:21.449990 20913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:21.455873 20923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:21.455861 20921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:21.455996 20913 server_base.cc:1034] running on GCE node
W20260809 23:09:21.455861 20920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:21.456351 20913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:21.456573 20913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:21.457744 20913 hybrid_clock.cc:648] HybridClock initialized: now 1786316961457724 us; error 42 us; skew 500 ppm
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:21.460721 20913 init.cc:377] Logged in from keytab as oryx/127.8.141.195@KRBTEST.COM (short username oryx)
I20260809 23:09:21.461230 20913 webserver.cc:460] Webserver started at http://127.8.141.195:42395/ using document root <none> and password file <none>
I20260809 23:09:21.461521 20913 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:21.461585 20913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:21.463132 20913 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:21.464228 20945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.464605 20913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:21.464708 20913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5"
format_stamp: "Formatted at 2026-08-09 23:09:20 on dist-test-slave-j2vl"
server_key: "b1e6a8599056413c1e4b3fbf75099b71"
server_key_iv: "fe21840156269917f641902e1c0e65b5"
server_key_version: "encryptionkey@0"
I20260809 23:09:21.465132 20913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:21.481046 20913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:21.483522 20913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:21.483688 20913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:21.483999 20913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:21.484406 20913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:21.484462 20913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.484505 20913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:21.484534 20913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:21.490604 20913 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:44743
I20260809 23:09:21.490650 21118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:44743 every 8 connection(s)
I20260809 23:09:21.491021 20913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:21.494203 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 20913
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.252@KRBTEST.COM
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.253@KRBTEST.COM
Aug 09 23:09:21 dist-test-slave-j2vl krb5kdc[18960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316961, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:21.504535 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.492158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:39613 (local address 127.8.141.252:34189)
0809 23:09:21.492204 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.492206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.492337 (+ 131us) server_negotiation.cc:407] Connection header received
0809 23:09:21.492899 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.492901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.492913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.492969 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.494028 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.494797 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.496413 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.496758 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.496804 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.500999 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.501001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.501001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.501010 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.503163 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503571 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.503572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.503573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.503637 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503856 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.503857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.503857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.503983 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.504042 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.504326 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.504433 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:09:21.504683 21127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.491876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.8.141.195:39613)
0809 23:09:21.492240 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.492250 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.492786 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.493039 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.493043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.493703 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.493916 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.493921 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.494911 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.494912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.496256 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.496263 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.496910 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.496910 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.496961 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.497757 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.497772 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.500896 (+ 3124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.503260 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.503261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.503490 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.503711 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.503712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.503774 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.504091 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.504092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.504136 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.504221 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.504400 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":92,"thread_start_us":102,"threads_started":1}
I20260809 23:09:21.505033 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.492093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:55829 (local address 127.8.141.253:41403)
0809 23:09:21.492141 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.492144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.492374 (+ 230us) server_negotiation.cc:407] Connection header received
0809 23:09:21.492959 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.492960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.492975 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.493009 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.494039 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.494853 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.496339 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.496648 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.496713 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.500707 (+ 3994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.500709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.500709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.500718 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.502596 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503171 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.503173 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.503173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.503240 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503566 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.503567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.503568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.503713 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.503774 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.504833 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.504923 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:21.505236 21128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.492026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.8.141.195:55829)
0809 23:09:21.492315 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.492325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.492822 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.493117 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.493117 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.493721 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.493917 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.493923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.494981 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.494982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.496255 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.496258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.496729 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.496730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.496870 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.497912 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.497917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.500577 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.502693 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.502703 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.502706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.503074 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.503332 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.503333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.503486 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.503805 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.503813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.503997 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.504111 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.505113 (+ 1002us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":88,"thread_start_us":65,"threads_started":1}
I20260809 23:09:21.505327 21120 heartbeater.cc:347] Connected to a master server at 127.8.141.252:34189
I20260809 23:09:21.505458 21120 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.505609 21124 heartbeater.cc:347] Connected to a master server at 127.8.141.253:41403
I20260809 23:09:21.505656 21124 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.505676 21120 heartbeater.cc:511] Master 127.8.141.252:34189 requested a full tablet report, sending...
I20260809 23:09:21.505715 21124 heartbeater.cc:511] Master 127.8.141.253:41403 requested a full tablet report, sending...
I20260809 23:09:21.506114 19094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.491868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:51883 (local address 127.8.141.254:37657)
0809 23:09:21.491959 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.491961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.492343 (+ 382us) server_negotiation.cc:407] Connection header received
0809 23:09:21.493119 (+ 776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.493120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.493136 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.493196 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.495391 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.496198 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.497418 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.497740 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.497804 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.501052 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.501054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.501054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.501062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.503704 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.504089 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.504090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.504091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.504154 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.504630 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.504631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.504631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.504798 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.504869 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.505912 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.506033 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 23:09:21.506196 20072 ts_manager.cc:194] Registered new tserver with Master: 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743)
I20260809 23:09:21.506201 20392 ts_manager.cc:194] Registered new tserver with Master: 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743)
I20260809 23:09:21.506603 21129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.491874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:51883)
0809 23:09:21.492179 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.492228 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.492822 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.493216 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.493217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.493725 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:21.493917 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.493923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.496477 (+ 2554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.496477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.497296 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.497299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.497934 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.497934 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.498007 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.498474 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:21.498479 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:21.500955 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:21.503805 (+ 2850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.503806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.503806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.504002 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.504222 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:21.504223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:21.504223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:21.504304 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:21.505064 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:21.505065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:21.505109 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.505175 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.506247 (+ 1072us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":91,"threads_started":1}
I20260809 23:09:21.506986 21121 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37657
I20260809 23:09:21.507045 21121 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:21.507126 21121 heartbeater.cc:511] Master 127.8.141.254:37657 requested a full tablet report, sending...
I20260809 23:09:21.507396 19037 ts_manager.cc:194] Re-registered known tserver with Master: 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743)
I20260809 23:09:21.507754 19037 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:51883
I20260809 23:09:21.511930 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.498036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.8.141.253:41403)
0809 23:09:21.499717 (+ 1681us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.499719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.499726 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:09:21.499748 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.499749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.499760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.499894 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.501818 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.502613 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.503715 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.504038 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.504099 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.508646 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.508648 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.508649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.508656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.510396 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.510780 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.510781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.510781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.510847 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.511090 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.511091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.511091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.511263 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.511371 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.511587 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.511816 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1658,"thread_start_us":398,"threads_started":1}
I20260809 23:09:21.512230 21141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.496310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48838 (local address 127.8.141.254:37657)
0809 23:09:21.497491 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.497493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.497499 (+ 6us) server_negotiation.cc:407] Connection header received
0809 23:09:21.497516 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.497517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.497530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.497592 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.499712 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.500592 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.506777 (+ 6185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.507087 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.507170 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.508762 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.508763 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.508764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.508771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.510744 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.511157 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.511158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.511158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.511227 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.511462 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.511463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.511463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.511608 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.512004 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.512053 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.512140 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1142,"thread_start_us":76,"threads_started":1}
I20260809 23:09:21.515383 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.498336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.8.141.252:34189)
0809 23:09:21.499214 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.499217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.499360 (+ 143us) server_negotiation.cc:407] Connection header received
0809 23:09:21.499382 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.499383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.499401 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.507253 (+ 7852us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.508294 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.509092 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.510303 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.510634 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.510711 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.512112 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.512113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.512113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.512120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.514015 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.514461 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.514462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.514462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.514534 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.514805 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.514806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.514807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.514964 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.515069 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.515169 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.515264 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":66,"threads_started":1}
I20260809 23:09:21.514446 19037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:09:21.517280 19037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:21.519424 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.515666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.8.141.252:34189)
0809 23:09:21.515736 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.515738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.515749 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:21.515775 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.515776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.515829 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.515947 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:21.516104 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.516953 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.518206 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.518977 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.519071 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.519208 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.519283 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.519347 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:09:21.519438 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.514851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.0.0.1:52450)
0809 23:09:21.514966 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.514979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.515030 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.515957 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.515959 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.515961 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:21.516018 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.516021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.517050 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.517051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.518120 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.518124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.519116 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.519118 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.519193 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.519199 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.519275 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.519334 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260809 23:09:21.519896 20413 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 23:09:21.520596 21159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:21.520705 21159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:21.520133 21160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb0bc160f904210b3c30373b00b6cd2. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: NON_VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: true } } }
I20260809 23:09:21.520793 21160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: This master's current role is: LEARNER
I20260809 23:09:21.522181 21163 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: attempting to promote NON_VOTER 33ad56731b30420dad50f4e3af1555ea to VOTER
I20260809 23:09:21.522353 21163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:21.523129 20413 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 LEARNER]: Refusing update from remote peer 6bb0bc160f904210b3c30373b00b6cd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260809 23:09:21.523449 20086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Refusing update from remote peer 6bb0bc160f904210b3c30373b00b6cd2: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260809 23:09:21.523473 21175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 23:09:21.524590 21175 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 23:09:21.524855 21159 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 33ad56731b30420dad50f4e3af1555ea (127.8.141.252) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.525274 20086 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 33ad56731b30420dad50f4e3af1555ea (127.8.141.252) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.526011 21159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.526098 21159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:21.526142 21159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.526196 21159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:21.526232 21159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.526278 21159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:21.526762 21185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.526901 21185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:09:21.526803 20413 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 33ad56731b30420dad50f4e3af1555ea (127.8.141.252) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.528071 21160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6bb0bc160f904210b3c30373b00b6cd2" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:21.528163 21160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:09:21.528970 21181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.523884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36881 (local address 127.0.0.1:39744)
0809 23:09:21.524223 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.524236 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.524287 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.524526 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.524527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.524530 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:21.524593 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.524595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.526359 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.526360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.527630 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.527633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.528677 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.528678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.528754 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.528760 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.528832 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.528883 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":58,"thread_start_us":81,"threads_started":1}
I20260809 23:09:21.528978 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.524094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39744 (local address 127.8.141.194:36881)
0809 23:09:21.524324 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.524330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.524341 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:21.524388 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.524390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.524409 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.524746 (+ 337us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:21.524823 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.526049 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.527737 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.528566 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.528673 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.528772 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.528850 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.528892 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":133,"threads_started":1}
I20260809 23:09:21.529170 21150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.523778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:44743 (local address 127.0.0.1:36932)
0809 23:09:21.523950 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.523966 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.524021 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.524628 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.524629 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.524631 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:21.524681 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.524683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.526407 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.526408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.527632 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.527635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.528673 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.528674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.528758 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.528765 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.528801 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.528858 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":7}
I20260809 23:09:21.529222 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.524057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.8.141.195:44743)
0809 23:09:21.524431 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.524436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.524447 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:21.524490 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.524492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.524509 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.524766 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:21.524839 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.526263 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.527741 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.528555 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.528939 (+ 384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.529050 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.529081 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.529132 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":93,"threads_started":1}
I20260809 23:09:21.530056 20800 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4 (DEFAULT_TABLE table=test-table [id=03fb709832694888a1c68a22ee5275d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:21.529999 21018 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4 (DEFAULT_TABLE table=test-table [id=03fb709832694888a1c68a22ee5275d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:21.530452 21018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:21.530453 20800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:21.532088 21176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.524066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41449 (local address 127.0.0.1:34834)
0809 23:09:21.526531 (+ 2465us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.526542 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.526593 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.527306 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.527307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.527309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:21.527352 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.527354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.529084 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.529086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.530328 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.530331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.531461 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.531463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.531622 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.531628 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.531932 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.531992 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2446,"thread_start_us":88,"threads_started":1}
I20260809 23:09:21.532337 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.524335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.8.141.193:41449)
0809 23:09:21.527052 (+ 2717us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.527058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.527070 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:21.527114 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.527116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.527134 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.527223 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:21.527432 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.528574 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.530449 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.531324 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.531410 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.531491 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.531821 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.531889 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2667,"thread_start_us":87,"threads_started":1}
I20260809 23:09:21.533092 20576 tablet_service.cc:1467] Processing CreateTablet for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4 (DEFAULT_TABLE table=test-table [id=03fb709832694888a1c68a22ee5275d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:21.533556 20576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:21.534457 21186 tablet_bootstrap.cc:492] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Bootstrap starting.
I20260809 23:09:21.534900 21188 tablet_bootstrap.cc:492] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Bootstrap starting.
I20260809 23:09:21.535403 21186 tablet_bootstrap.cc:654] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:21.535764 21188 tablet_bootstrap.cc:654] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:21.535902 21186 log.cc:824] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:21.536192 21188 log.cc:824] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:21.536899 21186 tablet_bootstrap.cc:492] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: No bootstrap required, opened a new log
I20260809 23:09:21.536947 21186 ts_tablet_manager.cc:1397] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:09:21.537137 21188 tablet_bootstrap.cc:492] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: No bootstrap required, opened a new log
I20260809 23:09:21.537206 21188 ts_tablet_manager.cc:1397] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:09:21.537510 21194 tablet_bootstrap.cc:492] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Bootstrap starting.
I20260809 23:09:21.538219 21194 tablet_bootstrap.cc:654] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:21.538806 21194 log.cc:824] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:21.539717 21194 tablet_bootstrap.cc:492] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: No bootstrap required, opened a new log
I20260809 23:09:21.539793 21194 ts_tablet_manager.cc:1397] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:09:21.539953 21186 raft_consensus.cc:348] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.540129 21186 raft_consensus.cc:374] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:21.540169 21186 raft_consensus.cc:729] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0e10f9fb6e4aa684a80f8c1741f5d5, State: Initialized, Role: FOLLOWER
I20260809 23:09:21.540197 21188 raft_consensus.cc:348] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.540346 21186 consensus_queue.cc:260] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.540397 21188 raft_consensus.cc:374] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:21.540441 21188 raft_consensus.cc:729] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6512e359cf2642e4bb79c7f5605228df, State: Initialized, Role: FOLLOWER
I20260809 23:09:21.540570 21188 consensus_queue.cc:260] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.540634 21186 ts_tablet_manager.cc:1428] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 23:09:21.540792 21121 heartbeater.cc:503] Master 127.8.141.254:37657 was elected leader, sending a full tablet report...
I20260809 23:09:21.540867 21188 ts_tablet_manager.cc:1428] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 23:09:21.541404 20903 heartbeater.cc:503] Master 127.8.141.254:37657 was elected leader, sending a full tablet report...
I20260809 23:09:21.544168 21194 raft_consensus.cc:348] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.544307 21194 raft_consensus.cc:374] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:21.544329 21194 raft_consensus.cc:729] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bb9cb93f01487c80ff6bade86d713f, State: Initialized, Role: FOLLOWER
I20260809 23:09:21.544459 21194 consensus_queue.cc:260] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.544744 21194 ts_tablet_manager.cc:1428] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 23:09:21.544811 20687 heartbeater.cc:503] Master 127.8.141.254:37657 was elected leader, sending a full tablet report...
W20260809 23:09:21.574299 20908 tablet.cc:2369] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:09:21.630842 20691 tablet.cc:2369] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:09:21.742501 21126 tablet.cc:2369] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:09:21.747265 21200 raft_consensus.cc:482] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:21.747394 21200 raft_consensus.cc:504] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.747836 21200 leader_election.cc:288] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743), 6512e359cf2642e4bb79c7f5605228df (127.8.141.194:36881)
I20260809 23:09:21.751034 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.748106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:44743 (local address 127.8.141.193:42417)
0809 23:09:21.748277 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.748290 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.748353 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.748555 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.748556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.748559 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:21.748627 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.748630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.749384 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.749385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.750246 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.750249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.750815 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.750817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.750893 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.750902 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.750933 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.750968 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":48,"thread_start_us":116,"threads_started":1}
I20260809 23:09:21.751052 21178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.748243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:42417 (local address 127.8.141.195:44743)
0809 23:09:21.748328 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.748331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.748348 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:09:21.748433 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.748434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.748445 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.748536 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:21.748716 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.749270 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.750379 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.750730 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.750909 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.750919 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.750948 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.750983 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 23:09:21.751240 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.748372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36881 (local address 127.8.141.193:58731)
0809 23:09:21.748509 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:21.748520 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:21.748573 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:21.748717 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:21.748718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:21.748720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:21.748763 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.748765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.749441 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.749442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.750229 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:21.750232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.750942 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.750943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:21.751031 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.751035 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:21.751094 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:09:21.751139 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":19,"thread_start_us":71,"threads_started":1}
I20260809 23:09:21.751523 21183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.748527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:58731 (local address 127.8.141.194:36881)
0809 23:09:21.748590 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.748592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.748601 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:21.748632 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.748633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.748642 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.748699 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:21.748853 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.749325 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.750327 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.750734 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.750826 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.750844 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.751405 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.751455 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 23:09:21.751516 21050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb45905b39b4a3eb5a8c10e1ef185e4" candidate_uuid: "57bb9cb93f01487c80ff6bade86d713f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" is_pre_election: true
I20260809 23:09:21.751688 21050 raft_consensus.cc:2454] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb9cb93f01487c80ff6bade86d713f in term 0.
I20260809 23:09:21.751896 20833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb45905b39b4a3eb5a8c10e1ef185e4" candidate_uuid: "57bb9cb93f01487c80ff6bade86d713f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6512e359cf2642e4bb79c7f5605228df" is_pre_election: true
I20260809 23:09:21.752012 20505 leader_election.cc:302] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb9cb93f01487c80ff6bade86d713f, 9f0e10f9fb6e4aa684a80f8c1741f5d5; no voters:
I20260809 23:09:21.752070 20833 raft_consensus.cc:2454] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57bb9cb93f01487c80ff6bade86d713f in term 0.
I20260809 23:09:21.752182 21200 raft_consensus.cc:2781] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:21.752244 21200 raft_consensus.cc:482] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:21.752269 21200 raft_consensus.cc:3037] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:21.753024 21200 raft_consensus.cc:504] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.753165 21200 leader_election.cc:288] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [CANDIDATE]: Term 1 election: Requested vote from peers 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743), 6512e359cf2642e4bb79c7f5605228df (127.8.141.194:36881)
I20260809 23:09:21.753444 20833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb45905b39b4a3eb5a8c10e1ef185e4" candidate_uuid: "57bb9cb93f01487c80ff6bade86d713f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6512e359cf2642e4bb79c7f5605228df"
I20260809 23:09:21.753432 21050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0cb45905b39b4a3eb5a8c10e1ef185e4" candidate_uuid: "57bb9cb93f01487c80ff6bade86d713f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5"
I20260809 23:09:21.753533 21050 raft_consensus.cc:3037] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:21.753533 20833 raft_consensus.cc:3037] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:21.754582 20833 raft_consensus.cc:2454] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb9cb93f01487c80ff6bade86d713f in term 1.
I20260809 23:09:21.754582 21050 raft_consensus.cc:2454] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57bb9cb93f01487c80ff6bade86d713f in term 1.
I20260809 23:09:21.754781 20505 leader_election.cc:302] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb9cb93f01487c80ff6bade86d713f, 6512e359cf2642e4bb79c7f5605228df; no voters:
I20260809 23:09:21.754915 21200 raft_consensus.cc:2781] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:21.755040 21200 raft_consensus.cc:686] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 1 LEADER]: Becoming Leader. State: Replica: 57bb9cb93f01487c80ff6bade86d713f, State: Running, Role: LEADER
I20260809 23:09:21.755162 21200 consensus_queue.cc:237] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } }
I20260809 23:09:21.756140 19037 catalog_manager.cc:5495] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f reported cstate change: term changed from 0 to 1, leader changed from <none> to 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193). New cstate: current_term: 1 leader_uuid: "57bb9cb93f01487c80ff6bade86d713f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } health_report { overall_health: UNKNOWN } } }
I20260809 23:09:21.815886 21184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.813109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34850 (local address 127.8.141.193:41449)
0809 23:09:21.813190 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.813193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.813203 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:09:21.813254 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.813254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.813274 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.813407 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:21.813580 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.814117 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.814976 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.815195 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.815283 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.815497 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:21.815620 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:21.815729 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.815752 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.815775 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 23:09:21.818509 20833 raft_consensus.cc:1261] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 1 FOLLOWER]: Refusing update from remote peer 57bb9cb93f01487c80ff6bade86d713f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:21.818547 21050 raft_consensus.cc:1261] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Refusing update from remote peer 57bb9cb93f01487c80ff6bade86d713f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:21.819007 21203 consensus_queue.cc:1035] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:21.819195 21200 consensus_queue.cc:1035] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:21.822762 21206 mvcc.cc:205] Tried to move back new op lower bound from 7316754275604623360 to 7316754275349843968. Current Snapshot: MvccSnapshot[applied={T|T < 7316754275604623360}]
I20260809 23:09:21.822906 21209 mvcc.cc:205] Tried to move back new op lower bound from 7316754275604623360 to 7316754275349843968. Current Snapshot: MvccSnapshot[applied={T|T < 7316754275604623360}]
I20260809 23:09:21.823601 21208 mvcc.cc:205] Tried to move back new op lower bound from 7316754275604623360 to 7316754275349843968. Current Snapshot: MvccSnapshot[applied={T|T < 7316754275604623360}]
I20260809 23:09:21.825829 19036 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:21.825945 19036 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:21.827454 19036 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bb0bc160f904210b3c30373b00b6cd2: Sending DeleteTablet for 3 replicas of tablet 0cb45905b39b4a3eb5a8c10e1ef185e4
I20260809 23:09:21.827795 20576 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:21 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:34834
I20260809 23:09:21.827860 21018 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:21 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:36932
I20260809 23:09:21.827842 20800 tablet_service.cc:1514] Processing DeleteTablet for tablet 0cb45905b39b4a3eb5a8c10e1ef185e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:21 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:39744
I20260809 23:09:21.828224 21225 tablet_replica.cc:330] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: stopping tablet replica
I20260809 23:09:21.828382 21225 raft_consensus.cc:2229] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:21.828562 21225 raft_consensus.cc:2258] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:21.829135 21225 ts_tablet_manager.cc:1905] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:21.829421 21223 tablet_replica.cc:330] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: stopping tablet replica
I20260809 23:09:21.829530 21223 raft_consensus.cc:2229] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:21.829576 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 18980
I20260809 23:09:21.829628 21223 raft_consensus.cc:2258] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:21.838559 21223 ts_tablet_manager.cc:1905] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:21.839838 21225 ts_tablet_manager.cc:1918] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:21.839944 21225 log.cc:1196] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/0cb45905b39b4a3eb5a8c10e1ef185e4
I20260809 23:09:21.840232 21225 ts_tablet_manager.cc:1939] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 6512e359cf2642e4bb79c7f5605228df: Deleting consensus metadata
I20260809 23:09:21.840569 21223 ts_tablet_manager.cc:1918] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:21.840754 21223 log.cc:1196] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/0cb45905b39b4a3eb5a8c10e1ef185e4
I20260809 23:09:21.841068 21223 ts_tablet_manager.cc:1939] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Deleting consensus metadata
I20260809 23:09:21.841595 21228 tablet_replica.cc:330] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: stopping tablet replica
I20260809 23:09:21.841734 21228 raft_consensus.cc:2229] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:21.841902 21228 raft_consensus.cc:2258] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:21.842446 21228 ts_tablet_manager.cc:1905] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:21.844120 21228 ts_tablet_manager.cc:1918] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:21.844201 21228 log.cc:1196] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/0cb45905b39b4a3eb5a8c10e1ef185e4
I20260809 23:09:21.844491 21228 ts_tablet_manager.cc:1939] T 0cb45905b39b4a3eb5a8c10e1ef185e4 P 57bb9cb93f01487c80ff6bade86d713f: Deleting consensus metadata
W20260809 23:09:21.846715 20504 connection.cc:535] server connection from 127.0.0.1:34834 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 23:09:21.846980 20727 connection.cc:535] server connection from 127.0.0.1:39744 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 23:09:21.847137 20951 connection.cc:535] server connection from 127.0.0.1:36932 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:09:21.849882 21144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.828672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52464 (local address 127.8.141.252:34189)
0809 23:09:21.828718 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.828720 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.840768 (+ 12048us) server_negotiation.cc:407] Connection header received
0809 23:09:21.840917 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.840919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.840933 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.840968 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.841899 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.842675 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.844095 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.844423 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.844475 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.846451 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.846453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.846453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.846462 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.848217 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.848604 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.848605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.848606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.848674 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.848904 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.848905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.848905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.849021 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.849135 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.849671 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.849770 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:09:21.857558 20698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.849858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.8.141.252:34189)
0809 23:09:21.849903 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.849905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.850070 (+ 165us) server_negotiation.cc:407] Connection header received
0809 23:09:21.850165 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.850166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.850176 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.850205 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.851028 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.851793 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.852675 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.853000 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.853076 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.854497 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.854499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.854499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.854507 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.856171 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.856621 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.856623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.856623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.856690 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.856933 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.856934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.856934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.857101 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.857198 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.857320 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.857428 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:21.858588 21143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:21.849496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57690 (local address 127.8.141.253:41403)
0809 23:09:21.849578 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:21.849580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:21.849749 (+ 169us) server_negotiation.cc:407] Connection header received
0809 23:09:21.849863 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:21.849864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:21.849880 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:21.849919 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:21.851917 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.852662 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.853619 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:21.853950 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:21.854018 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:21.855413 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:21.855415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:21.855416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:21.855424 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:21.857111 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.857747 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.857748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.857749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.857813 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:21.858033 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:21.858034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:21.858035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:21.858171 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:21.858250 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:21.858445 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 23:09:21.858526 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 23:09:22.824425 21244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:22.824095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:46343)
0809 23:09:22.824259 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:22.824357 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260809 23:09:22.824570 21245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:22.824150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:37165)
0809 23:09:22.824443 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:22.824520 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":65,"threads_started":1}
W20260809 23:09:22.824781 20687 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:37657 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
W20260809 23:09:22.824869 21121 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:37657 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:22.825003 21246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:22.824150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:53537)
0809 23:09:22.824597 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:22.824949 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":68,"threads_started":1}
W20260809 23:09:22.825230 20903 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:37657 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:23.328109 21247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bb0bc160f904210b3c30373b00b6cd2)
I20260809 23:09:23.328262 21247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:23.328544 21247 leader_election.cc:288] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657), 34efe63281b04edca6d53f0e00394a28 (127.8.141.253:41403)
I20260809 23:09:23.329003 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.328747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49286)
0809 23:09:23.328871 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.328955 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
W20260809 23:09:23.329193 20377 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:23.329239 21262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6bb0bc160f904210b3c30373b00b6cd2)
I20260809 23:09:23.329377 21262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:23.329667 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.329557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49296)
0809 23:09:23.329597 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.329633 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 23:09:23.329842 20377 leader_election.cc:334] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657): Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:23.329890 21262 leader_election.cc:288] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657), 33ad56731b30420dad50f4e3af1555ea (127.8.141.252:34189)
I20260809 23:09:23.330025 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.329714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49306)
0809 23:09:23.329858 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.329905 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
W20260809 23:09:23.330155 20051 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:23.330624 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.330503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49312)
0809 23:09:23.330543 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.330586 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 23:09:23.330724 20051 leader_election.cc:334] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657): Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:23.332878 21251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.328754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:41403 (local address 127.0.0.1:57698)
0809 23:09:23.328894 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.328908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:23.328972 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:23.329212 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:23.329214 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:23.329216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:23.329303 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:23.329306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.330649 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.330650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:23.331835 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:23.331839 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.332580 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.332581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:23.332655 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:23.332660 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:09:23.332724 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 23:09:23.332781 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260809 23:09:23.333415 21259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.328861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57698 (local address 127.8.141.253:41403)
0809 23:09:23.329005 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:23.329007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:23.329021 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:09:23.329054 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:23.329055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:23.329068 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:23.329146 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:23.329441 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.330546 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.331937 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.332487 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.332565 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:23.332579 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:23.333249 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 23:09:23.333302 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":116,"thread_start_us":67,"threads_started":1}
I20260809 23:09:23.333930 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.329777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:34189 (local address 127.0.0.1:52484)
0809 23:09:23.330110 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.330120 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:23.330175 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:23.330478 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:23.330479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:23.330483 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:23.330537 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:23.330539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.331462 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.331464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:23.332744 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:23.332747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.333582 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.333583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:23.333697 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:23.333735 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:09:23.333793 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:09:23.333859 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":79,"threads_started":1}
I20260809 23:09:23.333930 21268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.329927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52484 (local address 127.8.141.252:34189)
0809 23:09:23.330353 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:23.330356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:23.330365 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:23.330389 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:23.330390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:23.330399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:23.330471 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:23.330621 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.331356 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.332856 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:23.333473 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:23.333554 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:23.333565 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:23.333796 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 23:09:23.333853 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":118,"threads_started":1}
I20260809 23:09:23.334296 20086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33ad56731b30420dad50f4e3af1555ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34efe63281b04edca6d53f0e00394a28" is_pre_election: true
I20260809 23:09:23.334285 20413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34efe63281b04edca6d53f0e00394a28" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "33ad56731b30420dad50f4e3af1555ea" is_pre_election: true
I20260809 23:09:23.334420 20413 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34efe63281b04edca6d53f0e00394a28 in term 1.
I20260809 23:09:23.334416 20086 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33ad56731b30420dad50f4e3af1555ea in term 1.
I20260809 23:09:23.334663 20050 leader_election.cc:302] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33ad56731b30420dad50f4e3af1555ea, 34efe63281b04edca6d53f0e00394a28; no voters: 6bb0bc160f904210b3c30373b00b6cd2
I20260809 23:09:23.334689 20378 leader_election.cc:302] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33ad56731b30420dad50f4e3af1555ea, 34efe63281b04edca6d53f0e00394a28; no voters: 6bb0bc160f904210b3c30373b00b6cd2
I20260809 23:09:23.334815 21247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 23:09:23.334830 21262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 23:09:23.334865 21262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6bb0bc160f904210b3c30373b00b6cd2)
I20260809 23:09:23.334861 21247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6bb0bc160f904210b3c30373b00b6cd2)
I20260809 23:09:23.334903 21247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:09:23.334898 21262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:09:23.335624 21247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:23.335632 21262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:23.335783 21247 leader_election.cc:288] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 2 election: Requested vote from peers 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657), 34efe63281b04edca6d53f0e00394a28 (127.8.141.253:41403)
I20260809 23:09:23.335788 21262 leader_election.cc:288] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [CANDIDATE]: Term 2 election: Requested vote from peers 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657), 33ad56731b30420dad50f4e3af1555ea (127.8.141.252:34189)
I20260809 23:09:23.335983 21251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.335879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49314)
0809 23:09:23.335922 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.335949 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 23:09:23.336001 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.335880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49328)
0809 23:09:23.335945 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.335977 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 23:09:23.335992 20413 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "34efe63281b04edca6d53f0e00394a28" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "33ad56731b30420dad50f4e3af1555ea"
I20260809 23:09:23.336124 20413 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 34efe63281b04edca6d53f0e00394a28 in current term 2: Already voted for candidate 33ad56731b30420dad50f4e3af1555ea in this term.
I20260809 23:09:23.336275 20086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33ad56731b30420dad50f4e3af1555ea" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34efe63281b04edca6d53f0e00394a28"
I20260809 23:09:23.336313 21248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.336207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49332)
0809 23:09:23.336246 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.336279 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 23:09:23.336364 20086 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 33ad56731b30420dad50f4e3af1555ea in current term 2: Already voted for candidate 34efe63281b04edca6d53f0e00394a28 in this term.
I20260809 23:09:23.336405 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.336296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49342)
0809 23:09:23.336330 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.336357 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 23:09:23.336416 20377 leader_election.cc:334] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657): Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
W20260809 23:09:23.336493 20051 leader_election.cc:334] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657): Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:23.336553 20050 leader_election.cc:302] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34efe63281b04edca6d53f0e00394a28; no voters: 33ad56731b30420dad50f4e3af1555ea, 6bb0bc160f904210b3c30373b00b6cd2
I20260809 23:09:23.336629 21262 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 23:09:23.336710 20378 leader_election.cc:302] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 33ad56731b30420dad50f4e3af1555ea; no voters: 34efe63281b04edca6d53f0e00394a28, 6bb0bc160f904210b3c30373b00b6cd2
I20260809 23:09:23.336810 21247 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 23:09:23.825524 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.825151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:41177)
0809 23:09:23.825417 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.825464 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":91,"threads_started":1}
I20260809 23:09:23.825567 21273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.825258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:45457)
0809 23:09:23.825492 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.825523 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":127,"threads_started":1}
I20260809 23:09:23.825773 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:23.825539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:34167)
0809 23:09:23.825697 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:23.825733 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260809 23:09:24.826551 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:24.826311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:33479)
0809 23:09:24.826452 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:24.826497 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":51,"threads_started":1}
I20260809 23:09:24.826551 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:24.826295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:54715)
0809 23:09:24.826481 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:24.826512 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":77,"threads_started":1}
I20260809 23:09:24.826555 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:24.826254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:36769)
0809 23:09:24.826445 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:24.826494 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260809 23:09:24.981923 21280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:24.982038 21280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:24.982231 21280 leader_election.cc:288] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657), 34efe63281b04edca6d53f0e00394a28 (127.8.141.253:41403)
I20260809 23:09:24.982585 20086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33ad56731b30420dad50f4e3af1555ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34efe63281b04edca6d53f0e00394a28" is_pre_election: true
I20260809 23:09:24.982631 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:24.982397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49610)
0809 23:09:24.982544 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:24.982590 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260809 23:09:24.982687 20086 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33ad56731b30420dad50f4e3af1555ea in term 2.
I20260809 23:09:24.982923 20378 leader_election.cc:302] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33ad56731b30420dad50f4e3af1555ea, 34efe63281b04edca6d53f0e00394a28; no voters:
I20260809 23:09:24.983040 21280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 23:09:24.983083 21280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:24.983107 21280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:09:24.983202 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:24.983077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49624)
0809 23:09:24.983133 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:24.983169 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 23:09:24.983323 20377 leader_election.cc:334] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657): Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:24.983974 21280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 3 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:24.984130 21280 leader_election.cc:288] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 3 election: Requested vote from peers 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657), 34efe63281b04edca6d53f0e00394a28 (127.8.141.253:41403)
I20260809 23:09:24.984334 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:24.984198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49626)
0809 23:09:24.984259 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:24.984303 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 23:09:24.984306 20086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33ad56731b30420dad50f4e3af1555ea" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "34efe63281b04edca6d53f0e00394a28"
I20260809 23:09:24.984383 20086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:09:24.984717 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:24.984582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49628)
0809 23:09:24.984663 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:24.984695 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 23:09:24.984817 20377 leader_election.cc:334] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657): Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
I20260809 23:09:24.985191 20086 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33ad56731b30420dad50f4e3af1555ea in term 3.
I20260809 23:09:24.985448 20378 leader_election.cc:302] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 33ad56731b30420dad50f4e3af1555ea, 34efe63281b04edca6d53f0e00394a28; no voters: 6bb0bc160f904210b3c30373b00b6cd2
I20260809 23:09:24.985669 21280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 3 FOLLOWER]: Leader election won for term 3
I20260809 23:09:24.985996 21280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [term 3 LEADER]: Becoming Leader. State: Replica: 33ad56731b30420dad50f4e3af1555ea, State: Running, Role: LEADER
I20260809 23:09:24.986110 21280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } }
I20260809 23:09:24.986531 21285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ad56731b30420dad50f4e3af1555ea. Latest consensus state: current_term: 3 leader_uuid: "33ad56731b30420dad50f4e3af1555ea" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:24.986723 21285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:24.986912 21287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:24.987462 21287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:24.987537 21287 catalog_manager.cc:1259] Loaded cluster ID: 24cacf5cdfbf4981b675e1041fd70f32
I20260809 23:09:24.987555 21287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:24.987699 21287 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:24.987752 21287 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Loaded TSK: 0
I20260809 23:09:24.987855 21287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:09:25.036273 20392 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:09:25.038074 20392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:25.039268 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.039134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49642)
0809 23:09:25.039192 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.039229 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 23:09:25.039310 20086 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [term 3 FOLLOWER]: Refusing update from remote peer 33ad56731b30420dad50f4e3af1555ea: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 23:09:25.039675 21285 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 23:09:25.039896 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.039726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.0.0.1:49656)
0809 23:09:25.039797 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.039864 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 23:09:25.039986 20377 consensus_peers.cc:487] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea -> Peer 6bb0bc160f904210b3c30373b00b6cd2 (127.8.141.254:37657): Couldn't send request to peer 6bb0bc160f904210b3c30373b00b6cd2. Status: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 23:09:25.040488 21290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33ad56731b30420dad50f4e3af1555ea. Latest consensus state: current_term: 3 leader_uuid: "33ad56731b30420dad50f4e3af1555ea" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:25.040604 21290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:09:25.041077 21280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "33ad56731b30420dad50f4e3af1555ea" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:25.041193 21280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:25.041285 21289 mvcc.cc:205] Tried to move back new op lower bound from 7316754288798265344 to 7316754288583864320. Current Snapshot: MvccSnapshot[applied={T|T < 7316754288798265344}]
I20260809 23:09:25.041299 21292 mvcc.cc:205] Tried to move back new op lower bound from 7316754288798265344 to 7316754288583864320. Current Snapshot: MvccSnapshot[applied={T|T < 7316754288798265344}]
I20260809 23:09:25.041612 21290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "33ad56731b30420dad50f4e3af1555ea" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "6bb0bc160f904210b3c30373b00b6cd2" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37657 } } peers { permanent_uuid: "34efe63281b04edca6d53f0e00394a28" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 41403 } attrs { promote: false } } peers { permanent_uuid: "33ad56731b30420dad50f4e3af1555ea" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 34189 } attrs { promote: false } } }
I20260809 23:09:25.041708 21290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34efe63281b04edca6d53f0e00394a28 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:09:25.047045 21281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.043837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41449 (local address 127.0.0.1:34856)
0809 23:09:25.043891 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.043902 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:25.043954 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:25.044462 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:25.044462 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:25.044463 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:25.044552 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.044554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.045244 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.045245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.046141 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.046144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.046786 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.046787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.046853 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.046857 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:25.046909 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:09:25.046966 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":8}
I20260809 23:09:25.047359 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.043927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34856 (local address 127.8.141.193:41449)
0809 23:09:25.044309 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:25.044312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:25.044320 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:25.044347 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:25.044348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:25.044362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:25.044623 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:25.044663 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.045158 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.046241 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.046694 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.046786 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.046800 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:25.047224 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 23:09:25.047285 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":70,"threads_started":1}
I20260809 23:09:25.047712 20576 tablet_service.cc:1467] Processing CreateTablet for tablet 17e48b1876204c969c04025abe40ff5f (DEFAULT_TABLE table=test-table [id=c40eefe087214c39ad6e2cf229b60b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:25.047837 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.043838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36881 (local address 127.0.0.1:39756)
0809 23:09:25.044128 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.044139 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:25.044207 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:25.044460 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:25.044461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:25.044463 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:25.044552 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.044554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.045573 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.045574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.046805 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.046807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.047598 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.047599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.047667 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.047671 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:25.047718 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 23:09:25.047767 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":71,"thread_start_us":110,"threads_started":1}
I20260809 23:09:25.047837 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.044138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.8.141.195:44743)
0809 23:09:25.044246 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:25.044248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:25.044350 (+ 102us) server_negotiation.cc:407] Connection header received
0809 23:09:25.044483 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:25.044484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:25.044493 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:25.044537 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:25.044754 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.045475 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.046913 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.047436 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.047491 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.047505 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:25.047709 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 23:09:25.047754 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":49,"threads_started":1}
I20260809 23:09:25.047894 20576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e48b1876204c969c04025abe40ff5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:25.048096 21302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.044063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:44743 (local address 127.0.0.1:36946)
0809 23:09:25.044295 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.044306 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:25.044400 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:25.044624 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:25.044625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:25.044627 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:25.044675 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.044676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.045573 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.045574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.046805 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.046807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.047563 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.047563 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.047628 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.047632 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:25.047964 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 23:09:25.048020 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":54,"thread_start_us":81,"threads_started":1}
I20260809 23:09:25.048107 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.043927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.8.141.194:36881)
0809 23:09:25.044048 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:25.044050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:25.044247 (+ 197us) server_negotiation.cc:407] Connection header received
0809 23:09:25.044285 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:25.044286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:25.044367 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:25.044439 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:25.044632 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.045475 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.046880 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.047475 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.047546 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.047558 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:25.047930 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 23:09:25.048023 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
I20260809 23:09:25.049378 21305 tablet_bootstrap.cc:492] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f: Bootstrap starting.
I20260809 23:09:25.049656 20800 tablet_service.cc:1467] Processing CreateTablet for tablet 17e48b1876204c969c04025abe40ff5f (DEFAULT_TABLE table=test-table [id=c40eefe087214c39ad6e2cf229b60b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:25.049696 21018 tablet_service.cc:1467] Processing CreateTablet for tablet 17e48b1876204c969c04025abe40ff5f (DEFAULT_TABLE table=test-table [id=c40eefe087214c39ad6e2cf229b60b95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:25.049782 20800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e48b1876204c969c04025abe40ff5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:25.049832 21018 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17e48b1876204c969c04025abe40ff5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:25.049842 21305 tablet_bootstrap.cc:654] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:25.050485 21305 tablet_bootstrap.cc:492] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f: No bootstrap required, opened a new log
I20260809 23:09:25.050539 21305 ts_tablet_manager.cc:1397] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:25.050675 21305 raft_consensus.cc:348] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.050755 21305 raft_consensus.cc:374] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:25.050776 21305 raft_consensus.cc:729] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57bb9cb93f01487c80ff6bade86d713f, State: Initialized, Role: FOLLOWER
I20260809 23:09:25.050843 21305 consensus_queue.cc:260] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.051016 21305 ts_tablet_manager.cc:1428] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:09:25.051155 21307 tablet_bootstrap.cc:492] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Bootstrap starting.
I20260809 23:09:25.051571 21307 tablet_bootstrap.cc:654] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:25.051676 21312 tablet_bootstrap.cc:492] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: Bootstrap starting.
I20260809 23:09:25.051761 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.051371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:40067)
0809 23:09:25.051689 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.051729 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295}
I20260809 23:09:25.052125 21312 tablet_bootstrap.cc:654] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:25.052402 21307 tablet_bootstrap.cc:492] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: No bootstrap required, opened a new log
I20260809 23:09:25.052457 21307 ts_tablet_manager.cc:1397] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:25.052497 20685 heartbeater.cc:503] Master 127.8.141.252:34189 was elected leader, sending a full tablet report...
I20260809 23:09:25.052599 21307 raft_consensus.cc:348] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.052687 21307 raft_consensus.cc:374] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:25.052721 21307 raft_consensus.cc:729] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0e10f9fb6e4aa684a80f8c1741f5d5, State: Initialized, Role: FOLLOWER
I20260809 23:09:25.052790 21307 consensus_queue.cc:260] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.052878 21312 tablet_bootstrap.cc:492] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: No bootstrap required, opened a new log
I20260809 23:09:25.052932 21312 ts_tablet_manager.cc:1397] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:25.053006 21307 ts_tablet_manager.cc:1428] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:25.053268 21312 raft_consensus.cc:348] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.053373 21312 raft_consensus.cc:374] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:25.053375 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.053164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:37117)
0809 23:09:25.053258 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.053306 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 23:09:25.053400 21312 raft_consensus.cc:729] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6512e359cf2642e4bb79c7f5605228df, State: Initialized, Role: FOLLOWER
I20260809 23:09:25.053544 21120 heartbeater.cc:503] Master 127.8.141.252:34189 was elected leader, sending a full tablet report...
I20260809 23:09:25.053567 21312 consensus_queue.cc:260] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.053814 21312 ts_tablet_manager.cc:1428] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:25.053959 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.053858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:59579)
0809 23:09:25.053901 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.053928 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 23:09:25.054134 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.054038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:50095)
0809 23:09:25.054080 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.054104 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 23:09:25.054538 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.054422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:37201)
0809 23:09:25.054465 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.054501 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 23:09:25.054926 20902 heartbeater.cc:503] Master 127.8.141.252:34189 was elected leader, sending a full tablet report...
I20260809 23:09:25.055147 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.054981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:45205)
0809 23:09:25.055089 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.055118 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 23:09:25.209232 21322 raft_consensus.cc:482] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:25.209409 21322 raft_consensus.cc:504] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.209877 21322 leader_election.cc:288] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743), 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193:41449)
I20260809 23:09:25.214299 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.210022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:44743 (local address 127.8.141.194:37467)
0809 23:09:25.210074 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.210089 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:25.210152 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:25.210333 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:25.210335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:25.210337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:25.210406 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.210408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.211545 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.211546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.213084 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.213088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.213974 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.213977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.214060 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.214067 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:09:25.214140 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:09:25.214195 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":118}
I20260809 23:09:25.214320 21300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.210154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:45423 (local address 127.8.141.193:41449)
0809 23:09:25.210239 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:25.210241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:25.210392 (+ 151us) server_negotiation.cc:407] Connection header received
0809 23:09:25.210429 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:25.210429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:25.210470 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:25.210536 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:25.210675 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.211452 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.213234 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.213855 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.213935 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.213948 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:25.214190 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 23:09:25.214246 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 23:09:25.214321 21304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.210104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:37467 (local address 127.8.141.195:44743)
0809 23:09:25.210157 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:25.210160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:25.210175 (+ 15us) server_negotiation.cc:407] Connection header received
0809 23:09:25.210213 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:25.210214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:25.210226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:25.210315 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:25.210505 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.211381 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.213238 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.213845 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.213935 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.213948 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:25.214190 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 23:09:25.214258 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:09:25.214300 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.210088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41449 (local address 127.8.141.194:45423)
0809 23:09:25.210301 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.210311 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:25.210383 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:25.210546 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:25.210547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:25.210548 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:25.210581 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.210582 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.211548 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.211549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.213084 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:25.213088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.213999 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.214000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:25.214060 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.214068 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 23:09:25.214140 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 23:09:25.214195 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":48,"thread_start_us":155,"threads_started":1}
I20260809 23:09:25.214627 21050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e48b1876204c969c04025abe40ff5f" candidate_uuid: "6512e359cf2642e4bb79c7f5605228df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" is_pre_election: true
I20260809 23:09:25.214658 20609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e48b1876204c969c04025abe40ff5f" candidate_uuid: "6512e359cf2642e4bb79c7f5605228df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57bb9cb93f01487c80ff6bade86d713f" is_pre_election: true
I20260809 23:09:25.214787 21050 raft_consensus.cc:2454] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6512e359cf2642e4bb79c7f5605228df in term 0.
I20260809 23:09:25.214833 20609 raft_consensus.cc:2454] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6512e359cf2642e4bb79c7f5605228df in term 0.
I20260809 23:09:25.214991 20731 leader_election.cc:302] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6512e359cf2642e4bb79c7f5605228df, 9f0e10f9fb6e4aa684a80f8c1741f5d5; no voters:
I20260809 23:09:25.215142 21322 raft_consensus.cc:2781] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:25.215183 21322 raft_consensus.cc:482] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:25.215205 21322 raft_consensus.cc:3037] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:25.216001 21322 raft_consensus.cc:504] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.216163 21322 leader_election.cc:288] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [CANDIDATE]: Term 1 election: Requested vote from peers 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743), 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193:41449)
I20260809 23:09:25.216331 20609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e48b1876204c969c04025abe40ff5f" candidate_uuid: "6512e359cf2642e4bb79c7f5605228df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57bb9cb93f01487c80ff6bade86d713f"
I20260809 23:09:25.216331 21050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17e48b1876204c969c04025abe40ff5f" candidate_uuid: "6512e359cf2642e4bb79c7f5605228df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5"
I20260809 23:09:25.216434 21050 raft_consensus.cc:3037] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:25.216434 20609 raft_consensus.cc:3037] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:25.217298 21050 raft_consensus.cc:2454] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6512e359cf2642e4bb79c7f5605228df in term 1.
I20260809 23:09:25.217398 20609 raft_consensus.cc:2454] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6512e359cf2642e4bb79c7f5605228df in term 1.
I20260809 23:09:25.217545 20727 leader_election.cc:302] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57bb9cb93f01487c80ff6bade86d713f, 6512e359cf2642e4bb79c7f5605228df; no voters:
I20260809 23:09:25.217669 21322 raft_consensus.cc:2781] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:25.217815 21322 raft_consensus.cc:686] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 1 LEADER]: Becoming Leader. State: Replica: 6512e359cf2642e4bb79c7f5605228df, State: Running, Role: LEADER
I20260809 23:09:25.217906 21322 consensus_queue.cc:237] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } }
I20260809 23:09:25.218166 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.218059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:59779)
0809 23:09:25.218101 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.218133 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 23:09:25.218720 20392 catalog_manager.cc:5495] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df reported cstate change: term changed from 0 to 1, leader changed from <none> to 6512e359cf2642e4bb79c7f5605228df (127.8.141.194). New cstate: current_term: 1 leader_uuid: "6512e359cf2642e4bb79c7f5605228df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6512e359cf2642e4bb79c7f5605228df" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 } health_report { overall_health: UNKNOWN } } }
I20260809 23:09:25.232682 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.229922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.8.141.194:36881)
0809 23:09:25.229998 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:25.230000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:25.230013 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:09:25.230140 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:25.230141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:25.230162 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:25.230246 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:25.230413 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.230947 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.231789 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:25.232007 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:25.232098 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:25.232299 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:25.232417 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:25.232509 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:25.232554 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 23:09:25.232580 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 23:09:25.234273 21050 raft_consensus.cc:1261] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Refusing update from remote peer 6512e359cf2642e4bb79c7f5605228df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:25.234313 20609 raft_consensus.cc:1261] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 1 FOLLOWER]: Refusing update from remote peer 6512e359cf2642e4bb79c7f5605228df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:25.234668 21325 consensus_queue.cc:1035] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [LEADER]: Connected to new peer: Peer: permanent_uuid: "57bb9cb93f01487c80ff6bade86d713f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:25.234874 21322 consensus_queue.cc:1035] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f0e10f9fb6e4aa684a80f8c1741f5d5" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:25.235741 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.235605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:40343)
0809 23:09:25.235660 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.235698 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 23:09:25.235882 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.235767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:34665)
0809 23:09:25.235824 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.235850 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 23:09:25.237514 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.237412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:58261)
0809 23:09:25.237449 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.237479 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 23:09:25.237849 21277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.237680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.195:59657)
0809 23:09:25.237786 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.237815 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260809 23:09:25.237893 21275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.237769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.193:54887)
0809 23:09:25.237834 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.237862 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 23:09:25.238541 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:25.238424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37657 (local address 127.8.141.194:35549)
0809 23:09:25.238481 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.238509 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:37657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 23:09:25.240751 20392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:25.240870 20392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:25.242475 20392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33ad56731b30420dad50f4e3af1555ea: Sending DeleteTablet for 3 replicas of tablet 17e48b1876204c969c04025abe40ff5f
I20260809 23:09:25.242805 20800 tablet_service.cc:1514] Processing DeleteTablet for tablet 17e48b1876204c969c04025abe40ff5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:25 UTC) from {username='oryx', principal='oryx/127.8.141.252@KRBTEST.COM'} at 127.0.0.1:39756
I20260809 23:09:25.242853 21018 tablet_service.cc:1514] Processing DeleteTablet for tablet 17e48b1876204c969c04025abe40ff5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:25 UTC) from {username='oryx', principal='oryx/127.8.141.252@KRBTEST.COM'} at 127.0.0.1:36946
I20260809 23:09:25.242962 20576 tablet_service.cc:1514] Processing DeleteTablet for tablet 17e48b1876204c969c04025abe40ff5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:25 UTC) from {username='oryx', principal='oryx/127.8.141.252@KRBTEST.COM'} at 127.0.0.1:34856
I20260809 23:09:25.243192 21355 tablet_replica.cc:330] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f: stopping tablet replica
I20260809 23:09:25.243273 21355 raft_consensus.cc:2229] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:25.243336 21355 raft_consensus.cc:2258] T 17e48b1876204c969c04025abe40ff5f P 57bb9cb93f01487c80ff6bade86d713f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:25.243580 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 20334
I20260809 23:09:25.252254 21353 tablet_replica.cc:330] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: stopping tablet replica
I20260809 23:09:25.252372 21353 raft_consensus.cc:2229] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:25.252506 21353 raft_consensus.cc:2258] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:25.252668 21354 tablet_replica.cc:330] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: stopping tablet replica
I20260809 23:09:25.252887 21354 raft_consensus.cc:2229] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:25.252986 21354 raft_consensus.cc:2258] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:25.253063 21353 ts_tablet_manager.cc:1905] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:25.253327 21354 ts_tablet_manager.cc:1905] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:25.255121 21354 ts_tablet_manager.cc:1918] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:25.255203 21354 log.cc:1196] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/17e48b1876204c969c04025abe40ff5f
I20260809 23:09:25.255481 21354 ts_tablet_manager.cc:1939] T 17e48b1876204c969c04025abe40ff5f P 9f0e10f9fb6e4aa684a80f8c1741f5d5: Deleting consensus metadata
I20260809 23:09:25.255674 21353 ts_tablet_manager.cc:1918] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:25.255795 21353 log.cc:1196] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/17e48b1876204c969c04025abe40ff5f
I20260809 23:09:25.255898 20382 catalog_manager.cc:4841] TS 9f0e10f9fb6e4aa684a80f8c1741f5d5 (127.8.141.195:44743): tablet 17e48b1876204c969c04025abe40ff5f (table test-table [id=c40eefe087214c39ad6e2cf229b60b95]) successfully deleted
I20260809 23:09:25.256160 21353 ts_tablet_manager.cc:1939] T 17e48b1876204c969c04025abe40ff5f P 6512e359cf2642e4bb79c7f5605228df: Deleting consensus metadata
I20260809 23:09:25.256569 20382 catalog_manager.cc:4841] TS 6512e359cf2642e4bb79c7f5605228df (127.8.141.194:36881): tablet 17e48b1876204c969c04025abe40ff5f (table test-table [id=c40eefe087214c39ad6e2cf229b60b95]) successfully deleted
W20260809 23:09:25.259567 21302 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:25.259403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41449 (local address 127.0.0.1:48236)
0809 23:09:25.259463 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:25.259495 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:41449: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 23:09:25.259729 20372 catalog_manager.cc:4584] TS 57bb9cb93f01487c80ff6bade86d713f (127.8.141.193:41449): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17e48b1876204c969c04025abe40ff5f: Network error: Client connection negotiation failed: client connection to 127.8.141.193:41449: connect: Connection reset by peer (error 104)
I20260809 23:09:25.259796 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 20700
I20260809 23:09:25.275975 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 20913
I20260809 23:09:25.296720 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 19847
I20260809 23:09:25.308740 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 20169
2026-08-09T23:09:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7252 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:25 dist-test-slave-j2vl krb5kdc[21363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:25 dist-test-slave-j2vl krb5kdc[21363](info): set up 2 sockets
Aug 09 23:09:25 dist-test-slave-j2vl krb5kdc[21363](info): commencing operation
krb5kdc: starting...
W20260809 23:09:26.664649 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:26 dist-test-slave-j2vl krb5kdc[21363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab.
Entry for principal oryx/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab.
I20260809 23:09:26.800961 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:38919
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab
--principal=oryx/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:38919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:26.892954 21383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:26.893270 21383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:26.893316 21383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:26.894739 21383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:26.894832 21383 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 23:09:26.894858 21383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:26.894881 21383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:26.894902 21383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:26.894956 21383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:26.896708 21383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:38919
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.254.keytab
--principal=oryx/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:38919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:26.897043 21383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:26.897310 21383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:26.903206 21392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:26.903239 21383 server_base.cc:1034] running on GCE node
W20260809 23:09:26.903214 21399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:26.903214 21396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:26.903999 21383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:26.904414 21383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:26.905584 21383 hybrid_clock.cc:648] HybridClock initialized: now 1786316966905565 us; error 51 us; skew 500 ppm
Aug 09 23:09:26 dist-test-slave-j2vl krb5kdc[21363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316966, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:26.909577 21383 init.cc:377] Logged in from keytab as oryx/127.8.141.254@KRBTEST.COM (short username oryx)
I20260809 23:09:26.910125 21383 webserver.cc:460] Webserver started at http://127.8.141.254:36963/ using document root <none> and password file <none>
I20260809 23:09:26.910399 21383 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:26.910475 21383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:26.910636 21383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:26.911782 21383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "960a3f31738e4b11a36d33f82eb32f46"
format_stamp: "Formatted at 2026-08-09 23:09:26 on dist-test-slave-j2vl"
server_key: "5462cd98fa16e4c1ca0a94c68f532f09"
server_key_iv: "a0c697bd14ef608b2c7a2f376a05cfd7"
server_key_version: "encryptionkey@0"
I20260809 23:09:26.912171 21383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "960a3f31738e4b11a36d33f82eb32f46"
format_stamp: "Formatted at 2026-08-09 23:09:26 on dist-test-slave-j2vl"
server_key: "5462cd98fa16e4c1ca0a94c68f532f09"
server_key_iv: "a0c697bd14ef608b2c7a2f376a05cfd7"
server_key_version: "encryptionkey@0"
I20260809 23:09:26.913774 21383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:26.914909 21406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:26.915217 21383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:26.915345 21383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "960a3f31738e4b11a36d33f82eb32f46"
format_stamp: "Formatted at 2026-08-09 23:09:26 on dist-test-slave-j2vl"
server_key: "5462cd98fa16e4c1ca0a94c68f532f09"
server_key_iv: "a0c697bd14ef608b2c7a2f376a05cfd7"
server_key_version: "encryptionkey@0"
I20260809 23:09:26.915457 21383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:26.934135 21383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:26.936854 21383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:26.937031 21383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:26.942971 21383 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:38919
I20260809 23:09:26.943073 21514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:38919 every 8 connection(s)
I20260809 23:09:26.943447 21383 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:26.944301 21516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:26.946816 21516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Bootstrap starting.
I20260809 23:09:26.947546 21516 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:26.947858 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 21383
I20260809 23:09:26.947901 21516 log.cc:824] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:26.947952 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:26.948176 8759 external_mini_cluster.cc:1428] Setting key 7e48e7b2d03cceebe020beeca5790523
I20260809 23:09:26.948854 21516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: No bootstrap required, opened a new log
I20260809 23:09:26.951895 21516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960a3f31738e4b11a36d33f82eb32f46" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38919 } }
I20260809 23:09:26.952104 21516 raft_consensus.cc:374] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:26.952131 21516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 960a3f31738e4b11a36d33f82eb32f46, State: Initialized, Role: FOLLOWER
I20260809 23:09:26.952383 21516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960a3f31738e4b11a36d33f82eb32f46" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38919 } }
I20260809 23:09:26.952533 21516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:26.952574 21516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:26.952620 21516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:26.953600 21516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960a3f31738e4b11a36d33f82eb32f46" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38919 } }
I20260809 23:09:26.953738 21516 leader_election.cc:302] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 960a3f31738e4b11a36d33f82eb32f46; no voters:
I20260809 23:09:26.954017 21516 leader_election.cc:288] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:26.954030 21526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:26.954180 21526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [term 1 LEADER]: Becoming Leader. State: Replica: 960a3f31738e4b11a36d33f82eb32f46, State: Running, Role: LEADER
I20260809 23:09:26.954298 21526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960a3f31738e4b11a36d33f82eb32f46" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38919 } }
I20260809 23:09:26.954348 21516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:26.954682 21529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "960a3f31738e4b11a36d33f82eb32f46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960a3f31738e4b11a36d33f82eb32f46" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38919 } } }
I20260809 23:09:26.954802 21529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:26.955026 21530 sys_catalog.cc:455] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 960a3f31738e4b11a36d33f82eb32f46. Latest consensus state: current_term: 1 leader_uuid: "960a3f31738e4b11a36d33f82eb32f46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "960a3f31738e4b11a36d33f82eb32f46" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38919 } } }
I20260809 23:09:26.955101 21530 sys_catalog.cc:458] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:26.955438 21537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:26.956092 21537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:09:26 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:26.958915 21537 catalog_manager.cc:1347] Generated new cluster ID: 2c013737900e47e4b016842719bd13d4
I20260809 23:09:26.958979 21537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:26.961511 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:26.948781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.8.141.254:38919)
0809 23:09:26.949019 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:26.949028 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:26.949056 (+ 28us) server_negotiation.cc:407] Connection header received
0809 23:09:26.949573 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:26.949581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:26.949633 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:26.949755 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:26.950765 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:26.952057 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:26.953489 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:26.954267 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:26.954884 (+ 617us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:26.957749 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:26.957756 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:26.957756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:26.957769 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:26.959567 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:26.959993 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:26.959997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:26.959997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:26.960097 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:26.960327 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:26.960328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:26.960329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:26.960595 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:26.960773 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:26.960854 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 23:09:26.961008 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
I20260809 23:09:26.969035 21537 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:26.969851 21537 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:26.977021 21537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Generated new TSK 0
I20260809 23:09:26.977252 21537 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab.
Entry for principal oryx/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab.
I20260809 23:09:27.013947 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38919
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab
--principal=oryx/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:27.102468 21560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:27.102689 21560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:27.102715 21560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:27.103925 21560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:27.103969 21560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:27.103991 21560 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:09:27.104009 21560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:27.105685 21560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab
--principal=oryx/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:38919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:27.105953 21560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:27.106215 21560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:27.111626 21576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:27.111634 21574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:27.111702 21571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:27.111899 21560 server_base.cc:1034] running on GCE node
I20260809 23:09:27.112264 21560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:27.112480 21560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:27.113651 21560 hybrid_clock.cc:648] HybridClock initialized: now 1786316967113629 us; error 45 us; skew 500 ppm
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:27.116376 21560 init.cc:377] Logged in from keytab as oryx/127.8.141.193@KRBTEST.COM (short username oryx)
I20260809 23:09:27.116889 21560 webserver.cc:460] Webserver started at http://127.8.141.193:34769/ using document root <none> and password file <none>
I20260809 23:09:27.117125 21560 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:27.117194 21560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:27.117399 21560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:27.118615 21560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "1bc3ad1fdeaf9963d82f3344f9d3c959"
server_key_iv: "1e26540aaf2428669c7b778fad8dfc1b"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.118995 21560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "1bc3ad1fdeaf9963d82f3344f9d3c959"
server_key_iv: "1e26540aaf2428669c7b778fad8dfc1b"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.120646 21560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:27.121632 21586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.121824 21560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:27.122066 21560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "1bc3ad1fdeaf9963d82f3344f9d3c959"
server_key_iv: "1e26540aaf2428669c7b778fad8dfc1b"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.122174 21560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:27.140205 21560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:27.142669 21560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:27.142868 21560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:27.143558 21560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:27.143610 21560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.143651 21560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:27.143666 21560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:27.151580 21560 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:35637
I20260809 23:09:27.151952 21560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:27.152447 21772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:35637 every 8 connection(s)
I20260809 23:09:27.154755 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.144048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38919 (local address 127.8.141.193:48199)
0809 23:09:27.144258 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.144313 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.145108 (+ 795us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.145556 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.145560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.145817 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:27.146043 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.146048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.146799 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.146800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.147763 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.147767 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.148458 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.148459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.148719 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.149564 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:27.149576 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:27.151477 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:27.153247 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.153254 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.153256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.153633 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.153852 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.153853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.153853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.153960 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.154158 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:27.154163 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:27.154281 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.154424 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.154581 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260809 23:09:27.155099 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.144076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:48199 (local address 127.8.141.254:38919)
0809 23:09:27.144171 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.144174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.144433 (+ 259us) server_negotiation.cc:407] Connection header received
0809 23:09:27.145289 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.145290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.145305 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.145461 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.146151 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.146686 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.147869 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.148312 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.148399 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.151588 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.151589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.151590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.151598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.153171 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.153720 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.153721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.153722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.153787 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.154002 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.154002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.154002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.154106 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.154153 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.154940 (+ 787us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.155033 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:09:27.155544 21774 heartbeater.cc:347] Connected to a master server at 127.8.141.254:38919
I20260809 23:09:27.155633 21774 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:27.155829 21774 heartbeater.cc:511] Master 127.8.141.254:38919 requested a full tablet report, sending...
I20260809 23:09:27.156466 21435 ts_manager.cc:194] Registered new tserver with Master: a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637)
I20260809 23:09:27.157444 21435 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:48199
I20260809 23:09:27.160912 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 21560
I20260809 23:09:27.161042 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:27.161255 8759 external_mini_cluster.cc:1428] Setting key 31e98735f485b349f205196ed3f9e373
I20260809 23:09:27.162667 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.156710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:58671 (local address 127.8.141.254:38919)
0809 23:09:27.156757 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.156758 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.156848 (+ 90us) server_negotiation.cc:407] Connection header received
0809 23:09:27.156901 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.156902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.156910 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.156952 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.157285 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.158069 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.158731 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.158913 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.158970 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.159891 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.159892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.159892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.159898 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.161532 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.161894 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.161895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.161895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.161959 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.162194 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.162194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.162195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.162301 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.162354 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.162539 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.162603 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:09:27.162910 21781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.156621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38919 (local address 127.8.141.193:58671)
0809 23:09:27.156772 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.156784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.156845 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.156965 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.156966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.157161 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:27.157225 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.157226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.158144 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.158145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.158657 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.158659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.159001 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.159003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.159062 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.159479 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:27.159484 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:27.159794 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:27.161618 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.161619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.161620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.161817 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.162033 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.162034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.162035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.162107 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.162376 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:27.162377 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:27.162432 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.162509 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.162626 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for oryx/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab.
Entry for principal oryx/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab.
I20260809 23:09:27.193811 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38919
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab
--principal=oryx/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:27.285990 21784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:27.286269 21784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:27.286319 21784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:27.287616 21784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:27.287694 21784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:27.287752 21784 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:09:27.287809 21784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:27.289551 21784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab
--principal=oryx/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:38919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:27.289863 21784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:27.290133 21784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:27.296393 21791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:27.296370 21794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:27.296403 21784 server_base.cc:1034] running on GCE node
W20260809 23:09:27.296370 21796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:27.296972 21784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:27.297240 21784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:27.298429 21784 hybrid_clock.cc:648] HybridClock initialized: now 1786316967298437 us; error 51 us; skew 500 ppm
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:27.301302 21784 init.cc:377] Logged in from keytab as oryx/127.8.141.194@KRBTEST.COM (short username oryx)
I20260809 23:09:27.301849 21784 webserver.cc:460] Webserver started at http://127.8.141.194:35607/ using document root <none> and password file <none>
I20260809 23:09:27.302093 21784 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:27.302162 21784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:27.302321 21784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:27.303413 21784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "c6c6bf71d8a749058e70ac7810c4ef4c"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "177ddec75f07080a0b65221b48a897e4"
server_key_iv: "7edb77757b7dfeab0ee38b3c2a2e1e8f"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.303800 21784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "c6c6bf71d8a749058e70ac7810c4ef4c"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "177ddec75f07080a0b65221b48a897e4"
server_key_iv: "7edb77757b7dfeab0ee38b3c2a2e1e8f"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.305269 21784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:27.306149 21811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.306394 21784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:27.306525 21784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "c6c6bf71d8a749058e70ac7810c4ef4c"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "177ddec75f07080a0b65221b48a897e4"
server_key_iv: "7edb77757b7dfeab0ee38b3c2a2e1e8f"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.306690 21784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:27.323592 21784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:27.325949 21784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:27.326128 21784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:27.326694 21784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:27.326763 21784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.326800 21784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:27.326855 21784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.334360 21784 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:36165
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:27.334472 21993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:36165 every 8 connection(s)
I20260809 23:09:27.334836 21784 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:27.338790 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.327377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:32871 (local address 127.8.141.254:38919)
0809 23:09:27.327430 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.327437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.327656 (+ 219us) server_negotiation.cc:407] Connection header received
0809 23:09:27.328197 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.328199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.328214 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.328292 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.329182 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.330077 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.331460 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.331807 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.331900 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.335072 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.335074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.335075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.335085 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.336698 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.337554 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.337555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.337556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.337628 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.337991 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.337992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.337992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.338133 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.338202 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.338620 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.338715 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:27.338948 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.327366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38919 (local address 127.8.141.194:32871)
0809 23:09:27.327537 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.327575 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.328091 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.328382 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.328386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.328766 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:27.329067 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.329074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.330176 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.330177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.331356 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.331360 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.331934 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.331935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.332103 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.332753 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:27.332778 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:27.334893 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:27.336995 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.337004 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.337005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.337256 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.337739 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.337740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.337740 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.337881 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.338284 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:27.338293 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:27.338451 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.338560 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.338705 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260809 23:09:27.339659 21996 heartbeater.cc:347] Connected to a master server at 127.8.141.254:38919
I20260809 23:09:27.339792 21996 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:27.340102 21996 heartbeater.cc:511] Master 127.8.141.254:38919 requested a full tablet report, sending...
I20260809 23:09:27.340276 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 21784
I20260809 23:09:27.340399 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:27.340583 21436 ts_manager.cc:194] Registered new tserver with Master: c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165)
I20260809 23:09:27.340703 8759 external_mini_cluster.cc:1428] Setting key 3d57f4ed752d2220214f08316282bdce
I20260809 23:09:27.341128 21436 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:32871
I20260809 23:09:27.347684 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.341232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:35173 (local address 127.8.141.254:38919)
0809 23:09:27.341301 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.341303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.341311 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:27.341333 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.341334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.341367 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.341428 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.341848 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.342324 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.343045 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.343234 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.343309 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.344366 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.344368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.344368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.344376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.346133 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.346581 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.346582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.346583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.346644 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.346919 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.346920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.346920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.347147 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.347250 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.347489 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.347581 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 23:09:27.348132 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.340735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38919 (local address 127.8.141.194:35173)
0809 23:09:27.340896 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.340909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.340966 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.341478 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.341480 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.341694 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:27.341749 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.341751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.342431 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.342432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.342947 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.342949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.343308 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.343309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.343380 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.343920 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:27.343924 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:27.344249 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:27.346228 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.346230 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.346230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.346479 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.346727 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.346728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.346729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.346809 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.347256 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:27.347257 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:27.347356 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.347459 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.347635 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for oryx/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab.
Entry for principal oryx/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab.
I20260809 23:09:27.374640 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:38919
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab
--principal=oryx/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:27.465217 22015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:27.465560 22015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:27.465607 22015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:27.466930 22015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:27.467013 22015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:27.467052 22015 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 23:09:27.467092 22015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:27.468772 22015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab
--principal=oryx/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.141.254:38919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:27.469080 22015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:27.469396 22015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:27.474946 22022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:27.474970 22021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:27.475020 22015 server_base.cc:1034] running on GCE node
W20260809 23:09:27.474970 22024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:27.475414 22015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:27.475684 22015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:27.476855 22015 hybrid_clock.cc:648] HybridClock initialized: now 1786316967476834 us; error 45 us; skew 500 ppm
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:27.479787 22015 init.cc:377] Logged in from keytab as oryx/127.8.141.195@KRBTEST.COM (short username oryx)
I20260809 23:09:27.480208 22015 webserver.cc:460] Webserver started at http://127.8.141.195:38265/ using document root <none> and password file <none>
I20260809 23:09:27.480412 22015 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:27.480463 22015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:27.480561 22015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:27.481753 22015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "8ae48c1cffcc4dea7a959591628619d9"
server_key_iv: "319469ada9ddde00797c632417d77cf4"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.482080 22015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "8ae48c1cffcc4dea7a959591628619d9"
server_key_iv: "319469ada9ddde00797c632417d77cf4"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.483474 22015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:27.484279 22036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.484550 22015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:27.484689 22015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4"
format_stamp: "Formatted at 2026-08-09 23:09:27 on dist-test-slave-j2vl"
server_key: "8ae48c1cffcc4dea7a959591628619d9"
server_key_iv: "319469ada9ddde00797c632417d77cf4"
server_key_version: "encryptionkey@0"
I20260809 23:09:27.484774 22015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:27.497201 22015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:27.499979 22015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:27.500157 22015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:27.500737 22015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:27.500808 22015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.500882 22015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:27.500926 22015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.508164 22015 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:42917
I20260809 23:09:27.508292 22227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:42917 every 8 connection(s)
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:27.508607 22015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:27.511175 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 22015
I20260809 23:09:27.511313 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:27.511554 8759 external_mini_cluster.cc:1428] Setting key a0cea636d5e667c050bfbfbb48ac33f3
I20260809 23:09:27.512428 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.501401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:36269 (local address 127.8.141.254:38919)
0809 23:09:27.501469 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.501471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.501681 (+ 210us) server_negotiation.cc:407] Connection header received
0809 23:09:27.502340 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.502341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.502356 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.502408 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.503354 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.504192 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.505944 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.506282 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.506383 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.509254 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.509256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.509256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.509264 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.510702 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.511200 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.511201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.511202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.511271 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.511622 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.511623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.511624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.511801 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.511875 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.512230 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.512330 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:09:27.512744 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.501336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38919 (local address 127.8.141.195:36269)
0809 23:09:27.501556 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.501593 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.502231 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.502494 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.502498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.502906 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:27.503232 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.503239 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.504333 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.504335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.505831 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.505835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.506384 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.506384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.506492 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.507068 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:27.507079 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:27.509120 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:27.510786 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.510793 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.510795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.511118 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.511379 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.511381 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.511381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.511546 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.511882 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:27.511890 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:27.512073 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.512190 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.512374 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
I20260809 23:09:27.513314 22228 heartbeater.cc:347] Connected to a master server at 127.8.141.254:38919
I20260809 23:09:27.513541 22228 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:27.513743 22228 heartbeater.cc:511] Master 127.8.141.254:38919 requested a full tablet report, sending...
I20260809 23:09:27.514178 21435 ts_manager.cc:194] Registered new tserver with Master: f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917)
I20260809 23:09:27.514658 21435 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:36269
I20260809 23:09:27.520821 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.514535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:45665 (local address 127.8.141.254:38919)
0809 23:09:27.514802 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.514804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.514813 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:27.514832 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.514833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.514844 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.514897 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.515294 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.515843 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.516633 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.516822 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.516884 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.517959 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.517961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.517961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.517969 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.519421 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.519881 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.519882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.519883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.519969 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.520203 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.520204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.520205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.520339 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.520442 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.520626 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.520709 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260809 23:09:27.520840 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.514472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38919 (local address 127.8.141.195:45665)
0809 23:09:27.514610 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.514620 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.514675 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.514917 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.514918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.515132 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:27.515206 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.515208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.515956 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.515957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.516531 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.516533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.516897 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.516898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.516952 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.517516 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:27.517521 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:27.517832 (+ 311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:27.519534 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.519535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.519535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.519754 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.520051 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.520052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.520053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.520126 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.520434 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:27.520435 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:27.520501 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.520583 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.520712 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":7,"thread_start_us":74,"threads_started":1}
I20260809 23:09:27.523784 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 23:09:27.548172 21525 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:27.542191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36392 (local address 127.8.141.254:38919)
0809 23:09:27.542268 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.542270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.542286 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:09:27.542346 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.542347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.542364 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.542434 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.543067 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.543580 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.544358 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.544553 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.544630 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.548068 (+ 3438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36392: BlockingRecv error: recv got EOF from 127.0.0.1:36392 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:27.556761 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.549085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.8.141.254:38919)
0809 23:09:27.549188 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.549190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.549296 (+ 106us) server_negotiation.cc:407] Connection header received
0809 23:09:27.549407 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.549408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.549419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.549495 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.549994 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.550481 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.551183 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.551378 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.551465 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.553833 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.553835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.553836 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.553843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.555461 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.555930 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.555931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.555932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.555992 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.556239 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.556240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.556241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.556377 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.556466 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.556611 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.556683 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:09:27.558743 21435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:09:27.560509 21435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:27.567957 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.563603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43622 (local address 127.8.141.193:35637)
0809 23:09:27.563810 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.563814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.563827 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:09:27.563873 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.563876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.563894 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.563984 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:27.564260 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.565393 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.566730 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.567461 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.567541 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.567620 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.567795 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.567844 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
I20260809 23:09:27.567961 22270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.563379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:35637 (local address 127.0.0.1:43622)
0809 23:09:27.563503 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.563513 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.563578 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.564062 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.564064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.564066 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:27.564158 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.564161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.565481 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.565482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.566628 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.566631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.567563 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.567565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.567660 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.567678 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.567769 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.567838 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":103,"thread_start_us":69,"threads_started":1}
I20260809 23:09:27.568176 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.563051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36165 (local address 127.0.0.1:53712)
0809 23:09:27.563413 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.563469 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.563593 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.563967 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.563971 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.563974 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:27.564052 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.564054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.565459 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.565460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.566975 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.566979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.567935 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.567937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.568019 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.568024 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.568082 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.568130 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":35,"thread_start_us":204,"threads_started":1}
I20260809 23:09:27.568470 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.563184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.8.141.194:36165)
0809 23:09:27.563392 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.563397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.563553 (+ 156us) server_negotiation.cc:407] Connection header received
0809 23:09:27.563719 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.563721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.563736 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.563828 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:27.564142 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.565314 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.567095 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.567756 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.567822 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.567898 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.568302 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.568356 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":117,"threads_started":1}
I20260809 23:09:27.569279 21889 tablet_service.cc:1467] Processing CreateTablet for tablet 17306c1a9bd74dbdade80017249f574d (DEFAULT_TABLE table=test-table [id=f6304f3e179e41b4af46576fb05c70d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.569285 21669 tablet_service.cc:1467] Processing CreateTablet for tablet 17306c1a9bd74dbdade80017249f574d (DEFAULT_TABLE table=test-table [id=f6304f3e179e41b4af46576fb05c70d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.569759 21889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17306c1a9bd74dbdade80017249f574d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.569758 21669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17306c1a9bd74dbdade80017249f574d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.569994 22275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.563093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42917 (local address 127.0.0.1:58324)
0809 23:09:27.563491 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.563508 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.563578 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.564076 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.564077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.564079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:27.564158 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.564160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.567054 (+ 2894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.567055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.568872 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.568875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.569749 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.569750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.569799 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.569802 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.569882 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.569922 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260809 23:09:27.570281 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.563313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58324 (local address 127.8.141.195:42917)
0809 23:09:27.563845 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.563849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.563860 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:27.563900 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.563903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.563921 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.564007 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:27.565691 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.566924 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.568978 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.569572 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.569640 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.569717 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.570133 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.570178 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":88,"threads_started":1}
I20260809 23:09:27.574149 22279 tablet_bootstrap.cc:492] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Bootstrap starting.
I20260809 23:09:27.574151 22113 tablet_service.cc:1467] Processing CreateTablet for tablet 17306c1a9bd74dbdade80017249f574d (DEFAULT_TABLE table=test-table [id=f6304f3e179e41b4af46576fb05c70d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.574585 22113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17306c1a9bd74dbdade80017249f574d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.574896 22279 tablet_bootstrap.cc:654] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.575244 22279 log.cc:824] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:27.575914 22281 tablet_bootstrap.cc:492] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Bootstrap starting.
I20260809 23:09:27.576135 22279 tablet_bootstrap.cc:492] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: No bootstrap required, opened a new log
I20260809 23:09:27.576215 22279 ts_tablet_manager.cc:1397] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:09:27.576671 22281 tablet_bootstrap.cc:654] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.577018 22281 log.cc:824] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:27.578035 22281 tablet_bootstrap.cc:492] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: No bootstrap required, opened a new log
I20260809 23:09:27.578118 22281 ts_tablet_manager.cc:1397] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:09:27.578488 22286 tablet_bootstrap.cc:492] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Bootstrap starting.
I20260809 23:09:27.578790 22279 raft_consensus.cc:348] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.578963 22279 raft_consensus.cc:374] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.579008 22279 raft_consensus.cc:729] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c6bf71d8a749058e70ac7810c4ef4c, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.579141 22279 consensus_queue.cc:260] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.579239 22286 tablet_bootstrap.cc:654] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.579445 22279 ts_tablet_manager.cc:1428] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:09:27.579591 21996 heartbeater.cc:503] Master 127.8.141.254:38919 was elected leader, sending a full tablet report...
I20260809 23:09:27.579618 22286 log.cc:824] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:27.580514 22286 tablet_bootstrap.cc:492] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: No bootstrap required, opened a new log
I20260809 23:09:27.580597 22286 ts_tablet_manager.cc:1397] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:09:27.580660 22281 raft_consensus.cc:348] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.580837 22281 raft_consensus.cc:374] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.580878 22281 raft_consensus.cc:729] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58cd9648b8c4c12999f45d4b281b3bc, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.581024 22281 consensus_queue.cc:260] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.581329 22281 ts_tablet_manager.cc:1428] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 23:09:27.581382 21774 heartbeater.cc:503] Master 127.8.141.254:38919 was elected leader, sending a full tablet report...
I20260809 23:09:27.582896 22286 raft_consensus.cc:348] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.583039 22286 raft_consensus.cc:374] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.583063 22286 raft_consensus.cc:729] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.583209 22286 consensus_queue.cc:260] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.583519 22286 ts_tablet_manager.cc:1428] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:09:27.583545 22228 heartbeater.cc:503] Master 127.8.141.254:38919 was elected leader, sending a full tablet report...
W20260809 23:09:27.585480 22000 tablet.cc:2369] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:09:27.652895 21775 tablet.cc:2369] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:09:27.696139 22290 raft_consensus.cc:482] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:27.696280 22290 raft_consensus.cc:504] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.696732 22290 leader_election.cc:288] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165), a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637)
I20260809 23:09:27.698727 22287 raft_consensus.cc:482] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:27.698870 22287 raft_consensus.cc:504] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.699259 22287 leader_election.cc:288] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917)
I20260809 23:09:27.700553 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.696977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36165 (local address 127.8.141.195:46129)
0809 23:09:27.697039 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.697052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.697102 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.697467 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.697469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.697471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:27.697544 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.697546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.698306 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.698308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.699435 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.699439 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.700224 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.700225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.700340 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.700343 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.700394 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.700440 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":6}
I20260809 23:09:27.700553 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.697313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:46299 (local address 127.8.141.193:35637)
0809 23:09:27.697407 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.697409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.697420 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:27.697510 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.697511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.697519 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.697587 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:27.697756 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.698247 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.699538 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.700117 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.700166 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.700176 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.700433 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.700479 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 23:09:27.700798 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.697113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:35637 (local address 127.8.141.195:46299)
0809 23:09:27.697305 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.697312 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.697407 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.697594 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.697595 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.697597 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:27.697671 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.697675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.698348 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.698349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.699417 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.699420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.700235 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.700236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.700332 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.700336 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.700672 (+ 336us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.700714 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":2,"thread_start_us":112,"threads_started":1}
I20260809 23:09:27.700846 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.697181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:46129 (local address 127.8.141.194:36165)
0809 23:09:27.697262 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.697265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.697277 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:27.697311 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.697312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.697322 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.697442 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:27.697629 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.698123 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.699553 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.700113 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.700179 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.700189 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.700730 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.700771 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:09:27.701148 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17306c1a9bd74dbdade80017249f574d" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" is_pre_election: true
I20260809 23:09:27.701228 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17306c1a9bd74dbdade80017249f574d" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" is_pre_election: true
I20260809 23:09:27.701287 21921 raft_consensus.cc:2454] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:27.701411 21693 raft_consensus.cc:2454] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:27.701732 22042 leader_election.cc:302] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c6bf71d8a749058e70ac7810c4ef4c, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:27.701920 22290 raft_consensus.cc:2781] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:27.701990 22290 raft_consensus.cc:482] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:27.702023 22290 raft_consensus.cc:3037] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.703223 22290 raft_consensus.cc:504] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.703579 22290 leader_election.cc:288] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Requested vote from peers c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165), a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637)
I20260809 23:09:27.703562 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17306c1a9bd74dbdade80017249f574d" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c"
I20260809 23:09:27.703647 21921 raft_consensus.cc:3037] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.703629 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17306c1a9bd74dbdade80017249f574d" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
I20260809 23:09:27.703725 21693 raft_consensus.cc:3037] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.703994 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.699804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:47043 (local address 127.8.141.195:42917)
0809 23:09:27.699844 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.699846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.699895 (+ 49us) server_negotiation.cc:407] Connection header received
0809 23:09:27.699993 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.699994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.700002 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.700030 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:27.700251 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.700980 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.703091 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.703553 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.703645 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.703657 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.703879 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.703924 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:09:27.703996 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.699473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42917 (local address 127.8.141.194:47043)
0809 23:09:27.699805 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.699814 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.699922 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.700088 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.700088 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.700090 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:27.700173 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.700175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.701280 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.701281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.702999 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.703001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.703675 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.703676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.703763 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.703769 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.703836 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.703882 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":122,"thread_start_us":102,"threads_started":1}
I20260809 23:09:27.704324 21824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.699415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:35637 (local address 127.8.141.194:38979)
0809 23:09:27.699469 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.699481 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.699516 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.700076 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.700077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.700079 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:27.700137 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.700139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.701443 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.701444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.702705 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.702709 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.704092 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.704092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.704163 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.704166 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.704201 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.704239 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":27}
I20260809 23:09:27.704620 21921 raft_consensus.cc:2454] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:27.704636 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.699626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:38979 (local address 127.8.141.193:35637)
0809 23:09:27.699914 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.699917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.699924 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:09:27.699947 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.699947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.699972 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.700203 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:27.700252 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.700937 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.702804 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.703405 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.704444 (+ 1039us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.704467 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.704489 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.704541 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":53,"threads_started":1}
I20260809 23:09:27.704671 21693 raft_consensus.cc:2454] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:27.704850 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17306c1a9bd74dbdade80017249f574d" candidate_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" is_pre_election: true
I20260809 23:09:27.704957 21693 raft_consensus.cc:2379] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c6c6bf71d8a749058e70ac7810c4ef4c in current term 1: Already voted for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in this term.
I20260809 23:09:27.704950 22042 leader_election.cc:302] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58cd9648b8c4c12999f45d4b281b3bc, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:27.705060 22147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17306c1a9bd74dbdade80017249f574d" candidate_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" is_pre_election: true
I20260809 23:09:27.705066 22290 raft_consensus.cc:2781] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:27.705320 22290 raft_consensus.cc:686] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Becoming Leader. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Running, Role: LEADER
I20260809 23:09:27.705423 22290 consensus_queue.cc:237] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.706151 21814 leader_election.cc:302] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6c6bf71d8a749058e70ac7810c4ef4c; no voters: a58cd9648b8c4c12999f45d4b281b3bc, f46740a9755a4a2d8a3ca3c4c9dbb0d4
I20260809 23:09:27.706303 22287 raft_consensus.cc:2726] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 23:09:27.706362 21435 catalog_manager.cc:5495] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195). New cstate: current_term: 1 leader_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260809 23:09:27.717592 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.714854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58334 (local address 127.8.141.195:42917)
0809 23:09:27.714931 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.714933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.714944 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:27.715031 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.715031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.715054 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.715141 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:27.715309 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.715833 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.716667 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.716884 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.716966 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.717173 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:27.717282 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:27.717437 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.717463 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.717487 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:09:27.719931 21921 raft_consensus.cc:1261] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:27.719975 21693 raft_consensus.cc:1261] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:27.720292 22290 consensus_queue.cc:1035] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:27.720494 22295 consensus_queue.cc:1035] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:27.723428 22303 mvcc.cc:205] Tried to move back new op lower bound from 7316754299777089536 to 7316754299722051584. Current Snapshot: MvccSnapshot[applied={T|T < 7316754299777089536}]
I20260809 23:09:27.724026 22304 mvcc.cc:205] Tried to move back new op lower bound from 7316754299777089536 to 7316754299722051584. Current Snapshot: MvccSnapshot[applied={T|T < 7316754299777089536}]
I20260809 23:09:27.725991 21435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:27.726123 21435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:27.726934 21435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Sending DeleteTablet for 3 replicas of tablet 17306c1a9bd74dbdade80017249f574d
I20260809 23:09:27.727321 21889 tablet_service.cc:1514] Processing DeleteTablet for tablet 17306c1a9bd74dbdade80017249f574d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:27 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:53712
I20260809 23:09:27.727365 21669 tablet_service.cc:1514] Processing DeleteTablet for tablet 17306c1a9bd74dbdade80017249f574d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:27 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:43622
I20260809 23:09:27.727542 21435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 23:09:27.727600 22113 tablet_service.cc:1514] Processing DeleteTablet for tablet 17306c1a9bd74dbdade80017249f574d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:27 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:58324
W20260809 23:09:27.727695 21435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:27.727839 22318 tablet_replica.cc:330] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: stopping tablet replica
I20260809 23:09:27.727955 22318 raft_consensus.cc:2229] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:27.728080 22318 raft_consensus.cc:2258] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:27.728112 22316 tablet_replica.cc:330] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: stopping tablet replica
I20260809 23:09:27.728315 22316 raft_consensus.cc:2229] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:27.728479 22316 raft_consensus.cc:2258] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:27.728593 22318 ts_tablet_manager.cc:1905] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:27.729074 22316 ts_tablet_manager.cc:1905] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:27.729429 21889 tablet_service.cc:1467] Processing CreateTablet for tablet ac784c9432dd4ee0b9ad0698f5d08d43 (DEFAULT_TABLE table=test-table [id=abcf27422d41490ca3456919f521b493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.729562 21889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac784c9432dd4ee0b9ad0698f5d08d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.730398 22318 ts_tablet_manager.cc:1918] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:27.730487 22318 log.cc:1196] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/17306c1a9bd74dbdade80017249f574d
I20260809 23:09:27.730762 22318 ts_tablet_manager.cc:1939] T 17306c1a9bd74dbdade80017249f574d P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting consensus metadata
I20260809 23:09:27.730978 22279 tablet_bootstrap.cc:492] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: Bootstrap starting.
I20260809 23:09:27.731190 22113 tablet_service.cc:1467] Processing CreateTablet for tablet ac784c9432dd4ee0b9ad0698f5d08d43 (DEFAULT_TABLE table=test-table [id=abcf27422d41490ca3456919f521b493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.731290 22113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac784c9432dd4ee0b9ad0698f5d08d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.729318 21669 tablet_service.cc:1467] Processing CreateTablet for tablet ac784c9432dd4ee0b9ad0698f5d08d43 (DEFAULT_TABLE table=test-table [id=abcf27422d41490ca3456919f521b493]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.731391 21669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac784c9432dd4ee0b9ad0698f5d08d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.731423 22279 tablet_bootstrap.cc:654] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.732185 22316 ts_tablet_manager.cc:1918] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:27.732308 22316 log.cc:1196] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/17306c1a9bd74dbdade80017249f574d
I20260809 23:09:27.732511 22286 tablet_bootstrap.cc:492] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Bootstrap starting.
I20260809 23:09:27.732723 22281 tablet_bootstrap.cc:492] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: Bootstrap starting.
I20260809 23:09:27.732857 21412 catalog_manager.cc:4841] TS f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917): tablet 17306c1a9bd74dbdade80017249f574d (table test-table [id=f6304f3e179e41b4af46576fb05c70d6]) successfully deleted
I20260809 23:09:27.732949 22286 tablet_bootstrap.cc:654] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.733333 22315 tablet_replica.cc:330] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: stopping tablet replica
I20260809 23:09:27.733485 22315 raft_consensus.cc:2229] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:27.733537 22281 tablet_bootstrap.cc:654] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.733634 22315 raft_consensus.cc:2258] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:27.732714 22316 ts_tablet_manager.cc:1939] T 17306c1a9bd74dbdade80017249f574d P a58cd9648b8c4c12999f45d4b281b3bc: Deleting consensus metadata
I20260809 23:09:27.734130 22286 tablet_bootstrap.cc:492] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: No bootstrap required, opened a new log
I20260809 23:09:27.734189 22286 ts_tablet_manager.cc:1397] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:09:27.734306 22315 ts_tablet_manager.cc:1905] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:27.734350 22286 raft_consensus.cc:348] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.734432 22286 raft_consensus.cc:374] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.734460 22286 raft_consensus.cc:729] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.734509 22286 consensus_queue.cc:260] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.734624 22286 ts_tablet_manager.cc:1428] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.734760 21413 catalog_manager.cc:4841] TS a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637): tablet 17306c1a9bd74dbdade80017249f574d (table test-table [id=f6304f3e179e41b4af46576fb05c70d6]) successfully deleted
I20260809 23:09:27.735298 22281 tablet_bootstrap.cc:492] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: No bootstrap required, opened a new log
I20260809 23:09:27.735347 22281 ts_tablet_manager.cc:1397] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 23:09:27.735481 22281 raft_consensus.cc:348] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.735541 22281 raft_consensus.cc:374] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.735561 22281 raft_consensus.cc:729] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58cd9648b8c4c12999f45d4b281b3bc, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.735611 22281 consensus_queue.cc:260] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.735704 22281 ts_tablet_manager.cc:1428] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.736052 22279 tablet_bootstrap.cc:492] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: No bootstrap required, opened a new log
I20260809 23:09:27.736100 22279 ts_tablet_manager.cc:1397] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 23:09:27.736227 22279 raft_consensus.cc:348] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.736280 22279 raft_consensus.cc:374] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.736299 22279 raft_consensus.cc:729] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c6bf71d8a749058e70ac7810c4ef4c, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.736349 22279 consensus_queue.cc:260] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.736449 22279 ts_tablet_manager.cc:1428] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.736660 22315 ts_tablet_manager.cc:1918] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:27.736727 22315 log.cc:1196] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/17306c1a9bd74dbdade80017249f574d
I20260809 23:09:27.737118 22315 ts_tablet_manager.cc:1939] T 17306c1a9bd74dbdade80017249f574d P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting consensus metadata
I20260809 23:09:27.737565 21413 catalog_manager.cc:4841] TS c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165): tablet 17306c1a9bd74dbdade80017249f574d (table test-table [id=f6304f3e179e41b4af46576fb05c70d6]) successfully deleted
I20260809 23:09:27.752840 22290 raft_consensus.cc:482] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:27.753013 22290 raft_consensus.cc:504] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.753189 22290 leader_election.cc:288] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165)
I20260809 23:09:27.753500 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac784c9432dd4ee0b9ad0698f5d08d43" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" is_pre_election: true
I20260809 23:09:27.753618 21693 raft_consensus.cc:2454] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:27.753546 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac784c9432dd4ee0b9ad0698f5d08d43" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" is_pre_election: true
I20260809 23:09:27.753687 21921 raft_consensus.cc:2454] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:27.753787 22042 leader_election.cc:302] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58cd9648b8c4c12999f45d4b281b3bc, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:27.753890 22290 raft_consensus.cc:2781] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:27.753952 22290 raft_consensus.cc:482] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:27.753985 22290 raft_consensus.cc:3037] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.754971 22290 raft_consensus.cc:504] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.755116 22290 leader_election.cc:288] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Requested vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165)
I20260809 23:09:27.755364 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac784c9432dd4ee0b9ad0698f5d08d43" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
I20260809 23:09:27.755368 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac784c9432dd4ee0b9ad0698f5d08d43" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c"
I20260809 23:09:27.755472 21693 raft_consensus.cc:3037] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.755474 21921 raft_consensus.cc:3037] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.756371 21693 raft_consensus.cc:2454] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:27.756565 22042 leader_election.cc:302] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58cd9648b8c4c12999f45d4b281b3bc, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:27.756598 21921 raft_consensus.cc:2454] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:27.756659 22290 raft_consensus.cc:2781] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:27.756719 22290 raft_consensus.cc:686] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Becoming Leader. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Running, Role: LEADER
I20260809 23:09:27.756773 22290 consensus_queue.cc:237] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.757485 21436 catalog_manager.cc:5495] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195). New cstate: current_term: 1 leader_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } health_report { overall_health: HEALTHY } } }
W20260809 23:09:27.759198 22229 tablet.cc:2369] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316966, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.254@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:27.790798 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.782234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38919 (local address 127.0.0.1:36422)
0809 23:09:27.782432 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.782444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.782517 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.782671 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.782672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.783006 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:27.783067 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.783070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.783752 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.783754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.784286 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.784288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.784672 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.784673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.784742 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.785292 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:27.785297 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:27.787199 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:27.789464 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.789480 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.789482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.789701 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.789987 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:27.789988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:27.789988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:27.790072 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:27.790415 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:27.790424 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:27.790491 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.790570 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.790692 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":115,"threads_started":1}
I20260809 23:09:27.790895 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.782473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.8.141.254:38919)
0809 23:09:27.782540 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.782542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.782564 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:09:27.782593 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.782594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.782601 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.782666 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.783153 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.783669 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.784402 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.784587 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.784674 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.787303 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.787305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.787305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.787314 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.789371 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.789790 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.789791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.789791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.789866 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.790152 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.790153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.790153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.790313 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.790388 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.790618 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.790778 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":45}
I20260809 23:09:27.793396 21436 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:36422:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 23:09:27.793699 21436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:27.795416 21669 tablet_service.cc:1467] Processing CreateTablet for tablet 9e070c259d2542a3bb323a2ce7fe68fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87f404e6acf2463bae4e57a085df6b46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:09:27.795471 22113 tablet_service.cc:1467] Processing CreateTablet for tablet 9e070c259d2542a3bb323a2ce7fe68fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87f404e6acf2463bae4e57a085df6b46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:09:27.795504 21889 tablet_service.cc:1467] Processing CreateTablet for tablet 9e070c259d2542a3bb323a2ce7fe68fd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=87f404e6acf2463bae4e57a085df6b46]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 23:09:27.795616 21669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e070c259d2542a3bb323a2ce7fe68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.795634 22113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e070c259d2542a3bb323a2ce7fe68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.795632 21889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e070c259d2542a3bb323a2ce7fe68fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.797235 22279 tablet_bootstrap.cc:492] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: Bootstrap starting.
I20260809 23:09:27.797425 22281 tablet_bootstrap.cc:492] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: Bootstrap starting.
I20260809 23:09:27.797720 22279 tablet_bootstrap.cc:654] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.797876 22281 tablet_bootstrap.cc:654] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.798228 22286 tablet_bootstrap.cc:492] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Bootstrap starting.
I20260809 23:09:27.798564 22286 tablet_bootstrap.cc:654] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.798806 22279 tablet_bootstrap.cc:492] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: No bootstrap required, opened a new log
I20260809 23:09:27.798893 22279 ts_tablet_manager.cc:1397] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:09:27.799099 22279 raft_consensus.cc:348] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.799216 22279 raft_consensus.cc:374] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.799245 22279 raft_consensus.cc:729] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c6bf71d8a749058e70ac7810c4ef4c, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.799273 22286 tablet_bootstrap.cc:492] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: No bootstrap required, opened a new log
I20260809 23:09:27.799290 22279 consensus_queue.cc:260] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.799329 22286 ts_tablet_manager.cc:1397] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:27.799407 22279 ts_tablet_manager.cc:1428] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:09:27.799449 22286 raft_consensus.cc:348] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.799531 22286 raft_consensus.cc:374] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.799561 22286 raft_consensus.cc:729] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.799559 22287 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.799604 22286 consensus_queue.cc:260] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.799646 22287 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:09:27.799690 22286 ts_tablet_manager.cc:1428] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 23:09:27.799806 22308 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.799888 22308 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:09:27.800225 22281 tablet_bootstrap.cc:492] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: No bootstrap required, opened a new log
I20260809 23:09:27.800282 22281 ts_tablet_manager.cc:1397] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 23:09:27.800426 22281 raft_consensus.cc:348] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.800504 22281 raft_consensus.cc:374] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.800529 22281 raft_consensus.cc:729] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58cd9648b8c4c12999f45d4b281b3bc, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.800570 22281 consensus_queue.cc:260] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.800664 22281 ts_tablet_manager.cc:1428] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:09:27.800766 22289 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.800899 22289 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:09:27.855079 22287 raft_consensus.cc:482] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:27.855234 22287 raft_consensus.cc:504] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.855423 22287 leader_election.cc:288] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917)
I20260809 23:09:27.855679 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e070c259d2542a3bb323a2ce7fe68fd" candidate_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" is_pre_election: true
I20260809 23:09:27.855681 22147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e070c259d2542a3bb323a2ce7fe68fd" candidate_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" is_pre_election: true
I20260809 23:09:27.855803 21693 raft_consensus.cc:2454] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6c6bf71d8a749058e70ac7810c4ef4c in term 0.
I20260809 23:09:27.855827 22147 raft_consensus.cc:2454] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6c6bf71d8a749058e70ac7810c4ef4c in term 0.
I20260809 23:09:27.856004 21817 leader_election.cc:302] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58cd9648b8c4c12999f45d4b281b3bc, c6c6bf71d8a749058e70ac7810c4ef4c; no voters:
I20260809 23:09:27.856110 22287 raft_consensus.cc:2781] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:27.856143 22287 raft_consensus.cc:482] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:27.856172 22287 raft_consensus.cc:3037] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.857110 22287 raft_consensus.cc:504] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.857268 22287 leader_election.cc:288] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [CANDIDATE]: Term 1 election: Requested vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917)
I20260809 23:09:27.857479 22147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e070c259d2542a3bb323a2ce7fe68fd" candidate_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4"
I20260809 23:09:27.857515 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e070c259d2542a3bb323a2ce7fe68fd" candidate_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
I20260809 23:09:27.857564 22147 raft_consensus.cc:3037] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.857599 21693 raft_consensus.cc:3037] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:27.858193 21693 raft_consensus.cc:2454] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c6bf71d8a749058e70ac7810c4ef4c in term 1.
I20260809 23:09:27.858371 22147 raft_consensus.cc:2454] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6c6bf71d8a749058e70ac7810c4ef4c in term 1.
I20260809 23:09:27.858382 21817 leader_election.cc:302] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58cd9648b8c4c12999f45d4b281b3bc, c6c6bf71d8a749058e70ac7810c4ef4c; no voters:
I20260809 23:09:27.858469 22287 raft_consensus.cc:2781] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:27.858583 22287 raft_consensus.cc:686] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 LEADER]: Becoming Leader. State: Replica: c6c6bf71d8a749058e70ac7810c4ef4c, State: Running, Role: LEADER
I20260809 23:09:27.858655 22287 consensus_queue.cc:237] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.859200 22344 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: TxnStatusTablet state changed. Reason: New leader c6c6bf71d8a749058e70ac7810c4ef4c. Latest consensus state: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.859404 22344 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:09:27.859565 22346 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 23:09:27.859565 21436 catalog_manager.cc:5495] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c reported cstate change: term changed from 0 to 1, leader changed from <none> to c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194). New cstate: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } health_report { overall_health: UNKNOWN } } }
I20260809 23:09:27.859687 22346 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 23:09:27.867367 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.864058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53718 (local address 127.8.141.194:36165)
0809 23:09:27.864106 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.864108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.864119 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:27.864238 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.864239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.864260 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.864319 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:27.864536 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.865303 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.866476 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.866699 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.866765 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.867004 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:27.867117 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:27.867178 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.867267 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.867287 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:09:27.867413 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.863985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36165 (local address 127.0.0.1:53718)
0809 23:09:27.864073 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.864085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.864132 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.864374 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.864375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.864376 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:09:27.864446 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.864448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.865422 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.865423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.866346 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.866349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.866793 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.866795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.866872 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.866895 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:09:27.867189 (+ 294us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:09:27.867201 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.867263 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.867320 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 23:09:27.868881 21693 raft_consensus.cc:1261] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Refusing update from remote peer c6c6bf71d8a749058e70ac7810c4ef4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:27.868920 22147 raft_consensus.cc:1261] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Refusing update from remote peer c6c6bf71d8a749058e70ac7810c4ef4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:27.869176 22344 consensus_queue.cc:1035] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:27.869455 22344 consensus_queue.cc:1035] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:27.870152 22308 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: TxnStatusTablet state changed. Reason: New leader c6c6bf71d8a749058e70ac7810c4ef4c. Latest consensus state: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.870266 22308 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:09:27.870286 22289 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: TxnStatusTablet state changed. Reason: New leader c6c6bf71d8a749058e70ac7810c4ef4c. Latest consensus state: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.870389 22289 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:09:27.871315 22287 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.871431 22287 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:09:27.871498 22287 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.871572 22287 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P c6c6bf71d8a749058e70ac7810c4ef4c: This TxnStatusTablet replica's current role is: LEADER
I20260809 23:09:27.871891 22308 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.871990 22308 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P f46740a9755a4a2d8a3ca3c4c9dbb0d4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:09:27.872260 22289 tablet_replica.cc:439] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } } }
I20260809 23:09:27.872371 22289 tablet_replica.cc:442] T 9e070c259d2542a3bb323a2ce7fe68fd P a58cd9648b8c4c12999f45d4b281b3bc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 23:09:27.879176 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.876184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:47319 (local address 127.8.141.194:36165)
0809 23:09:27.876231 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.876234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.876272 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:09:27.876357 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.876358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.876369 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.876423 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:27.876597 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.877411 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.878229 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.878529 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.878599 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.878780 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:27.878816 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:27.878871 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.879065 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.879088 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:27.879179 22240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.876096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:36165 (local address 127.8.141.195:47319)
0809 23:09:27.876191 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.876206 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.876297 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.876467 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.876468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.876470 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:09:27.876518 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.876520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.877498 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.877499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.878143 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.878145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.878611 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.878613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.878667 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.878686 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:09:27.878948 (+ 262us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:09:27.878951 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.879048 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.879106 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 23:09:27.882038 21921 raft_consensus.cc:1261] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:27.882072 21693 raft_consensus.cc:1261] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:27.882285 22308 consensus_queue.cc:1035] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:27.882408 22290 consensus_queue.cc:1035] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:27.883692 22301 mvcc.cc:205] Tried to move back new op lower bound from 7316754300443029504 to 7316754299932205056. Current Snapshot: MvccSnapshot[applied={T|T < 7316754300443029504}]
I20260809 23:09:27.892460 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.889309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:35391 (local address 127.8.141.195:42917)
0809 23:09:27.889399 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.889401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.889412 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:27.889467 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.889468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.889479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.889544 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:27.889686 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.890460 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.891593 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.891905 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.891980 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.892189 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:27.892226 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:27.892280 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.892375 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.892399 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 23:09:27.892765 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.889085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42917 (local address 127.8.141.194:35391)
0809 23:09:27.889308 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:27.889321 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:27.889409 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:27.889546 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:27.889546 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:27.889548 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 23:09:27.889618 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.889619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.890592 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.890593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.891510 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:27.891513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.892002 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.892003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:27.892062 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.892081 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 23:09:27.892308 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 23:09:27.892311 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:27.892358 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 23:09:27.892410 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":138,"threads_started":1}
I20260809 23:09:27.902279 21436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:27.902421 21436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:27.903211 21436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Sending DeleteTablet for 3 replicas of tablet ac784c9432dd4ee0b9ad0698f5d08d43
I20260809 23:09:27.903537 21669 tablet_service.cc:1514] Processing DeleteTablet for tablet ac784c9432dd4ee0b9ad0698f5d08d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:27 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:43622
I20260809 23:09:27.903669 22113 tablet_service.cc:1514] Processing DeleteTablet for tablet ac784c9432dd4ee0b9ad0698f5d08d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:27 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:58324
I20260809 23:09:27.903683 22316 tablet_replica.cc:330] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: stopping tablet replica
I20260809 23:09:27.903752 22316 raft_consensus.cc:2229] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:27.903752 21889 tablet_service.cc:1514] Processing DeleteTablet for tablet ac784c9432dd4ee0b9ad0698f5d08d43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:27 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:53712
I20260809 23:09:27.903797 22316 raft_consensus.cc:2258] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:27.903857 22315 tablet_replica.cc:330] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: stopping tablet replica
I20260809 23:09:27.903877 22318 tablet_replica.cc:330] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: stopping tablet replica
I20260809 23:09:27.903924 22315 raft_consensus.cc:2229] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:27.903947 22318 raft_consensus.cc:2229] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:27.903971 22315 raft_consensus.cc:2258] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:27.904016 22318 raft_consensus.cc:2258] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:27.904320 22315 ts_tablet_manager.cc:1905] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:27.904395 22318 ts_tablet_manager.cc:1905] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:27.904407 22316 ts_tablet_manager.cc:1905] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:27.906342 22315 ts_tablet_manager.cc:1918] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:09:27.906340 22318 ts_tablet_manager.cc:1918] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:09:27.906412 22315 log.cc:1196] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/ac784c9432dd4ee0b9ad0698f5d08d43
I20260809 23:09:27.906430 22318 log.cc:1196] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/ac784c9432dd4ee0b9ad0698f5d08d43
I20260809 23:09:27.906649 22315 ts_tablet_manager.cc:1939] T ac784c9432dd4ee0b9ad0698f5d08d43 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting consensus metadata
I20260809 23:09:27.906687 22318 ts_tablet_manager.cc:1939] T ac784c9432dd4ee0b9ad0698f5d08d43 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting consensus metadata
I20260809 23:09:27.907363 21412 catalog_manager.cc:4841] TS f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917): tablet ac784c9432dd4ee0b9ad0698f5d08d43 (table test-table [id=abcf27422d41490ca3456919f521b493]) successfully deleted
I20260809 23:09:27.907420 21413 catalog_manager.cc:4841] TS c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165): tablet ac784c9432dd4ee0b9ad0698f5d08d43 (table test-table [id=abcf27422d41490ca3456919f521b493]) successfully deleted
I20260809 23:09:27.909888 22316 ts_tablet_manager.cc:1918] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:09:27.909982 22316 log.cc:1196] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/ac784c9432dd4ee0b9ad0698f5d08d43
I20260809 23:09:27.910236 22316 ts_tablet_manager.cc:1939] T ac784c9432dd4ee0b9ad0698f5d08d43 P a58cd9648b8c4c12999f45d4b281b3bc: Deleting consensus metadata
I20260809 23:09:27.910727 21413 catalog_manager.cc:4841] TS a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637): tablet ac784c9432dd4ee0b9ad0698f5d08d43 (table test-table [id=abcf27422d41490ca3456919f521b493]) successfully deleted
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 23:09:27.939071 21525 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:27.932031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36432 (local address 127.8.141.254:38919)
0809 23:09:27.932113 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.932115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.932232 (+ 117us) server_negotiation.cc:407] Connection header received
0809 23:09:27.932318 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.932320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.932333 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.932403 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.933094 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.933989 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.934875 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.935222 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.935308 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.938990 (+ 3682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36432: BlockingRecv error: recv got EOF from 127.0.0.1:36432 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 23:09:27 dist-test-slave-j2vl krb5kdc[21363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM
I20260809 23:09:27.948877 21525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:27.940285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36436 (local address 127.8.141.254:38919)
0809 23:09:27.940327 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:27.940329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:27.940407 (+ 78us) server_negotiation.cc:407] Connection header received
0809 23:09:27.940444 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:27.940444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:27.940456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:27.940528 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:27.941044 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.941926 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.942648 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:27.943017 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:27.943094 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:27.946051 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:27.946052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:27.946052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:27.946060 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:27.947634 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.948073 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.948074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.948075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.948140 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:27.948375 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:27.948376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:27.948376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:27.948487 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:27.948562 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:27.948736 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 23:09:27.948804 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:09:27.950728 21436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:09:27.950941 21436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:27.952687 21889 tablet_service.cc:1467] Processing CreateTablet for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 (DEFAULT_TABLE table=test-table [id=1c3f850ecba94e09a30f40f10b0f27b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.952695 22113 tablet_service.cc:1467] Processing CreateTablet for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 (DEFAULT_TABLE table=test-table [id=1c3f850ecba94e09a30f40f10b0f27b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.952689 21669 tablet_service.cc:1467] Processing CreateTablet for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 (DEFAULT_TABLE table=test-table [id=1c3f850ecba94e09a30f40f10b0f27b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:27.952872 21669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.952872 21889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.952872 22113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:27.954372 22286 tablet_bootstrap.cc:492] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Bootstrap starting.
I20260809 23:09:27.954680 22281 tablet_bootstrap.cc:492] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: Bootstrap starting.
I20260809 23:09:27.954779 22286 tablet_bootstrap.cc:654] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.955157 22281 tablet_bootstrap.cc:654] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.955564 22279 tablet_bootstrap.cc:492] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: Bootstrap starting.
I20260809 23:09:27.955593 22286 tablet_bootstrap.cc:492] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: No bootstrap required, opened a new log
I20260809 23:09:27.955649 22286 ts_tablet_manager.cc:1397] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:27.955786 22286 raft_consensus.cc:348] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.955866 22286 raft_consensus.cc:374] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.955890 22286 raft_consensus.cc:729] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.955920 22279 tablet_bootstrap.cc:654] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:27.955958 22286 consensus_queue.cc:260] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.956064 22286 ts_tablet_manager.cc:1428] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:27.956923 22279 tablet_bootstrap.cc:492] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: No bootstrap required, opened a new log
I20260809 23:09:27.956991 22279 ts_tablet_manager.cc:1397] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:27.957122 22279 raft_consensus.cc:348] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.957176 22279 raft_consensus.cc:374] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.957190 22279 raft_consensus.cc:729] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c6bf71d8a749058e70ac7810c4ef4c, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.957244 22279 consensus_queue.cc:260] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.957609 22281 tablet_bootstrap.cc:492] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: No bootstrap required, opened a new log
I20260809 23:09:27.957674 22281 ts_tablet_manager.cc:1397] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 23:09:27.957863 22281 raft_consensus.cc:348] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.957942 22281 raft_consensus.cc:374] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:27.957974 22281 raft_consensus.cc:729] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58cd9648b8c4c12999f45d4b281b3bc, State: Initialized, Role: FOLLOWER
I20260809 23:09:27.958027 22281 consensus_queue.cc:260] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:27.958171 22281 ts_tablet_manager.cc:1428] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:09:27.958586 22279 ts_tablet_manager.cc:1428] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 23:09:28.168844 22290 raft_consensus.cc:482] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:28.168982 22290 raft_consensus.cc:504] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.169169 22290 leader_election.cc:288] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165)
I20260809 23:09:28.169787 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e38c24cb5154a0dad4fd8222d3aa6f4" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" is_pre_election: true
I20260809 23:09:28.169787 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e38c24cb5154a0dad4fd8222d3aa6f4" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" is_pre_election: true
I20260809 23:09:28.169935 21693 raft_consensus.cc:2454] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:28.169936 21921 raft_consensus.cc:2454] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:28.170159 22042 leader_election.cc:302] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58cd9648b8c4c12999f45d4b281b3bc, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:28.170289 22290 raft_consensus.cc:2781] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:28.170325 22290 raft_consensus.cc:482] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:28.170358 22290 raft_consensus.cc:3037] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:28.171072 22290 raft_consensus.cc:504] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.171221 22290 leader_election.cc:288] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Requested vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165)
I20260809 23:09:28.171430 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e38c24cb5154a0dad4fd8222d3aa6f4" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
I20260809 23:09:28.171496 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e38c24cb5154a0dad4fd8222d3aa6f4" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c"
I20260809 23:09:28.171555 21693 raft_consensus.cc:3037] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:28.171562 21921 raft_consensus.cc:3037] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:28.172200 21921 raft_consensus.cc:2454] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:28.172314 21693 raft_consensus.cc:2454] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:28.172410 22042 leader_election.cc:302] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c6bf71d8a749058e70ac7810c4ef4c, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:28.172514 22290 raft_consensus.cc:2781] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:28.172564 22290 raft_consensus.cc:686] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Becoming Leader. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Running, Role: LEADER
I20260809 23:09:28.172623 22290 consensus_queue.cc:237] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.173324 21436 catalog_manager.cc:5495] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195). New cstate: current_term: 1 leader_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260809 23:09:28.188665 22274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:28.186157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.8.141.195:42917)
0809 23:09:28.186227 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:28.186229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:28.186245 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:09:28.186283 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:28.186284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:28.186294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:28.186356 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:28.186473 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:28.187025 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:28.187915 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:28.188125 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:28.188190 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:28.188335 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:28.188402 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:28.188463 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:28.188569 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 23:09:28.188597 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:09:28.189623 21921 raft_consensus.cc:1261] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:28.189651 21693 raft_consensus.cc:1261] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:28.189939 22308 consensus_queue.cc:1035] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:28.190068 22290 consensus_queue.cc:1035] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:28.193059 21436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:28.193163 21436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:28.193874 21436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Sending DeleteTablet for 3 replicas of tablet 9e38c24cb5154a0dad4fd8222d3aa6f4
I20260809 23:09:28.194156 22113 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:28 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:58324
I20260809 23:09:28.194278 21889 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:28 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:53712
I20260809 23:09:28.194289 22318 tablet_replica.cc:330] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: stopping tablet replica
I20260809 23:09:28.194379 22318 raft_consensus.cc:2229] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:28.194383 22315 tablet_replica.cc:330] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: stopping tablet replica
I20260809 23:09:28.194399 21669 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:28 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:43622
I20260809 23:09:28.194440 22315 raft_consensus.cc:2229] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:28.194459 22318 raft_consensus.cc:2258] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:28.194495 22315 raft_consensus.cc:2258] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:28.194490 22316 tablet_replica.cc:330] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: stopping tablet replica
I20260809 23:09:28.194545 22316 raft_consensus.cc:2229] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:28.194584 22316 raft_consensus.cc:2258] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:28.194784 22318 ts_tablet_manager.cc:1905] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:28.194841 21436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 23:09:28.194947 22316 ts_tablet_manager.cc:1905] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 23:09:28.194999 21436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:28.196513 21889 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8aa16648d44ae290ff7a23c6a04942 (DEFAULT_TABLE table=test-table [id=e3d06bb21fc041d7bce51c90b9015bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:28.196604 21669 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8aa16648d44ae290ff7a23c6a04942 (DEFAULT_TABLE table=test-table [id=e3d06bb21fc041d7bce51c90b9015bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:28.196655 21889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8aa16648d44ae290ff7a23c6a04942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:28.196771 22113 tablet_service.cc:1467] Processing CreateTablet for tablet 0f8aa16648d44ae290ff7a23c6a04942 (DEFAULT_TABLE table=test-table [id=e3d06bb21fc041d7bce51c90b9015bb6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:28.197067 22316 ts_tablet_manager.cc:1918] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:28.197146 22316 log.cc:1196] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/9e38c24cb5154a0dad4fd8222d3aa6f4
I20260809 23:09:28.197408 22316 ts_tablet_manager.cc:1939] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P a58cd9648b8c4c12999f45d4b281b3bc: Deleting consensus metadata
I20260809 23:09:28.197638 21669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8aa16648d44ae290ff7a23c6a04942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:28.198205 22279 tablet_bootstrap.cc:492] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: Bootstrap starting.
I20260809 23:09:28.198599 22279 tablet_bootstrap.cc:654] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:28.198725 22281 tablet_bootstrap.cc:492] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc: Bootstrap starting.
I20260809 23:09:28.198812 22318 ts_tablet_manager.cc:1918] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:28.198861 22318 log.cc:1196] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/9e38c24cb5154a0dad4fd8222d3aa6f4
I20260809 23:09:28.199095 22318 ts_tablet_manager.cc:1939] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting consensus metadata
I20260809 23:09:28.199193 22281 tablet_bootstrap.cc:654] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:28.199254 22113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f8aa16648d44ae290ff7a23c6a04942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:28.199841 22281 tablet_bootstrap.cc:492] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc: No bootstrap required, opened a new log
I20260809 23:09:28.199911 22281 ts_tablet_manager.cc:1397] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:28.200073 22281 raft_consensus.cc:348] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.200150 22281 raft_consensus.cc:374] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:28.200170 22281 raft_consensus.cc:729] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a58cd9648b8c4c12999f45d4b281b3bc, State: Initialized, Role: FOLLOWER
I20260809 23:09:28.200222 22281 consensus_queue.cc:260] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.200363 22281 ts_tablet_manager.cc:1428] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 23:09:28.200493 22286 tablet_bootstrap.cc:492] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Bootstrap starting.
I20260809 23:09:28.200733 21413 catalog_manager.cc:4841] TS a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637): tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 (table test-table [id=1c3f850ecba94e09a30f40f10b0f27b1]) successfully deleted
I20260809 23:09:28.200876 21412 catalog_manager.cc:4841] TS f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917): tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 (table test-table [id=1c3f850ecba94e09a30f40f10b0f27b1]) successfully deleted
I20260809 23:09:28.200923 22286 tablet_bootstrap.cc:654] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:28.201303 22279 tablet_bootstrap.cc:492] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: No bootstrap required, opened a new log
I20260809 23:09:28.201368 22279 ts_tablet_manager.cc:1397] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 23:09:28.201512 22279 raft_consensus.cc:348] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.201594 22286 tablet_bootstrap.cc:492] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: No bootstrap required, opened a new log
I20260809 23:09:28.201598 22279 raft_consensus.cc:374] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:28.201628 22286 ts_tablet_manager.cc:1397] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:28.201625 22279 raft_consensus.cc:729] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6c6bf71d8a749058e70ac7810c4ef4c, State: Initialized, Role: FOLLOWER
I20260809 23:09:28.201684 22279 consensus_queue.cc:260] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.201763 22286 raft_consensus.cc:348] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.201810 22279 ts_tablet_manager.cc:1428] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:28.201839 22286 raft_consensus.cc:374] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:28.201881 22286 raft_consensus.cc:729] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Initialized, Role: FOLLOWER
I20260809 23:09:28.201937 22286 consensus_queue.cc:260] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.202021 22315 ts_tablet_manager.cc:1905] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:28.202036 22286 ts_tablet_manager.cc:1428] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:28.203574 22315 ts_tablet_manager.cc:1918] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:28.203653 22315 log.cc:1196] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/9e38c24cb5154a0dad4fd8222d3aa6f4
I20260809 23:09:28.203925 22315 ts_tablet_manager.cc:1939] T 9e38c24cb5154a0dad4fd8222d3aa6f4 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting consensus metadata
I20260809 23:09:28.204273 21413 catalog_manager.cc:4841] TS c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165): tablet 9e38c24cb5154a0dad4fd8222d3aa6f4 (table test-table [id=1c3f850ecba94e09a30f40f10b0f27b1]) successfully deleted
I20260809 23:09:28.382786 22308 raft_consensus.cc:482] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:28.382903 22308 raft_consensus.cc:504] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.383111 22308 leader_election.cc:288] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165)
I20260809 23:09:28.383343 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8aa16648d44ae290ff7a23c6a04942" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" is_pre_election: true
I20260809 23:09:28.383457 21693 raft_consensus.cc:2454] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:28.383400 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8aa16648d44ae290ff7a23c6a04942" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" is_pre_election: true
I20260809 23:09:28.383538 21921 raft_consensus.cc:2454] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 0.
I20260809 23:09:28.383672 22042 leader_election.cc:302] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a58cd9648b8c4c12999f45d4b281b3bc, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:28.383808 22308 raft_consensus.cc:2781] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:28.383884 22308 raft_consensus.cc:482] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:28.383908 22308 raft_consensus.cc:3037] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:28.384651 22308 raft_consensus.cc:504] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.384793 22308 leader_election.cc:288] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Requested vote from peers a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637), c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165)
I20260809 23:09:28.384985 21693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8aa16648d44ae290ff7a23c6a04942" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a58cd9648b8c4c12999f45d4b281b3bc"
I20260809 23:09:28.385002 21921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f8aa16648d44ae290ff7a23c6a04942" candidate_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c"
I20260809 23:09:28.385075 21693 raft_consensus.cc:3037] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:28.385097 21921 raft_consensus.cc:3037] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:28.385798 21693 raft_consensus.cc:2454] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:28.385901 21921 raft_consensus.cc:2454] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f46740a9755a4a2d8a3ca3c4c9dbb0d4 in term 1.
I20260809 23:09:28.386152 22042 leader_election.cc:302] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6c6bf71d8a749058e70ac7810c4ef4c, f46740a9755a4a2d8a3ca3c4c9dbb0d4; no voters:
I20260809 23:09:28.386327 22308 raft_consensus.cc:2781] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:28.386492 22308 raft_consensus.cc:686] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Becoming Leader. State: Replica: f46740a9755a4a2d8a3ca3c4c9dbb0d4, State: Running, Role: LEADER
I20260809 23:09:28.386590 22308 consensus_queue.cc:237] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } }
I20260809 23:09:28.387629 21435 catalog_manager.cc:5495] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195). New cstate: current_term: 1 leader_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f46740a9755a4a2d8a3ca3c4c9dbb0d4" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42917 } health_report { overall_health: HEALTHY } } }
I20260809 23:09:28.432989 21921 raft_consensus.cc:1261] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:28.433022 21693 raft_consensus.cc:1261] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Refusing update from remote peer f46740a9755a4a2d8a3ca3c4c9dbb0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:28.433249 22290 consensus_queue.cc:1035] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a58cd9648b8c4c12999f45d4b281b3bc" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:28.433421 22308 consensus_queue.cc:1035] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6c6bf71d8a749058e70ac7810c4ef4c" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 36165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:28.443538 21436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:28.443655 21436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:28.444325 21436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 960a3f31738e4b11a36d33f82eb32f46: Sending DeleteTablet for 3 replicas of tablet 0f8aa16648d44ae290ff7a23c6a04942
I20260809 23:09:28.444628 22113 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8aa16648d44ae290ff7a23c6a04942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:28 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:58324
I20260809 23:09:28.444618 21669 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8aa16648d44ae290ff7a23c6a04942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:28 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:43622
I20260809 23:09:28.444727 22318 tablet_replica.cc:330] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: stopping tablet replica
I20260809 23:09:28.444727 22316 tablet_replica.cc:330] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc: stopping tablet replica
I20260809 23:09:28.444778 22318 raft_consensus.cc:2229] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:28.444780 22316 raft_consensus.cc:2229] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:28.444818 22316 raft_consensus.cc:2258] T 0f8aa16648d44ae290ff7a23c6a04942 P a58cd9648b8c4c12999f45d4b281b3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:28.444844 22318 raft_consensus.cc:2258] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:28.444986 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 21560
I20260809 23:09:28.445012 21889 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f8aa16648d44ae290ff7a23c6a04942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:28 UTC) from {username='oryx', principal='oryx/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:53712
I20260809 23:09:28.445101 22315 tablet_replica.cc:330] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: stopping tablet replica
I20260809 23:09:28.445153 22315 raft_consensus.cc:2229] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:28.445194 22315 raft_consensus.cc:2258] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:28.445186 22318 ts_tablet_manager.cc:1905] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:28.447968 22318 ts_tablet_manager.cc:1918] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:09:28.448052 22318 log.cc:1196] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/0f8aa16648d44ae290ff7a23c6a04942
I20260809 23:09:28.448292 22318 ts_tablet_manager.cc:1939] T 0f8aa16648d44ae290ff7a23c6a04942 P f46740a9755a4a2d8a3ca3c4c9dbb0d4: Deleting consensus metadata
I20260809 23:09:28.448441 22315 ts_tablet_manager.cc:1905] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:28.450523 22315 ts_tablet_manager.cc:1918] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 23:09:28.450613 22315 log.cc:1196] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/0f8aa16648d44ae290ff7a23c6a04942
I20260809 23:09:28.450870 22315 ts_tablet_manager.cc:1939] T 0f8aa16648d44ae290ff7a23c6a04942 P c6c6bf71d8a749058e70ac7810c4ef4c: Deleting consensus metadata
I20260809 23:09:28.451329 21413 catalog_manager.cc:4841] TS c6c6bf71d8a749058e70ac7810c4ef4c (127.8.141.194:36165): tablet 0f8aa16648d44ae290ff7a23c6a04942 (table test-table [id=e3d06bb21fc041d7bce51c90b9015bb6]) successfully deleted
I20260809 23:09:28.452337 21412 catalog_manager.cc:4841] TS f46740a9755a4a2d8a3ca3c4c9dbb0d4 (127.8.141.195:42917): tablet 0f8aa16648d44ae290ff7a23c6a04942 (table test-table [id=e3d06bb21fc041d7bce51c90b9015bb6]) successfully deleted
W20260809 23:09:28.459854 21413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:35637 (error 108)
I20260809 23:09:28.460264 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 21784
I20260809 23:09:28.463228 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:28.460453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:35637 (local address 127.0.0.1:43630)
0809 23:09:28.463100 (+ 2647us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:28.463167 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:35637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2623,"thread_start_us":39,"threads_started":1}
W20260809 23:09:28.463351 21413 catalog_manager.cc:4584] TS a58cd9648b8c4c12999f45d4b281b3bc (127.8.141.193:35637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f8aa16648d44ae290ff7a23c6a04942: Network error: Client connection negotiation failed: client connection to 127.8.141.193:35637: connect: Connection refused (error 111)
I20260809 23:09:28.480602 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 22015
I20260809 23:09:28.499265 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 21383
2026-08-09T23:09:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3194 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:28 dist-test-slave-j2vl krb5kdc[22438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:28 dist-test-slave-j2vl krb5kdc[22438](info): set up 2 sockets
Aug 09 23:09:28 dist-test-slave-j2vl krb5kdc[22438](info): commencing operation
krb5kdc: starting...
W20260809 23:09:29.827572 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.296s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:29 dist-test-slave-j2vl krb5kdc[22438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:29.978490 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41799
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:41799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:30.066087 22457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:30.066314 22457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:30.066339 22457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:30.067512 22457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:30.067557 22457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:30.067571 22457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:30.067584 22457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:30.067597 22457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:30.069311 22457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:41799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:30.069664 22457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:30.069938 22457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:30.075423 22465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:30.075506 22467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:30.075641 22457 server_base.cc:1034] running on GCE node
W20260809 23:09:30.075466 22464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:30.076067 22457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:30.076434 22457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:30.077661 22457 hybrid_clock.cc:648] HybridClock initialized: now 1786316970077648 us; error 42 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T23:09:30Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1578 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:30 dist-test-slave-j2vl krb5kdc[22475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:30 dist-test-slave-j2vl krb5kdc[22475](info): set up 2 sockets
Aug 09 23:09:30 dist-test-slave-j2vl krb5kdc[22475](info): commencing operation
krb5kdc: starting...
W20260809 23:09:31.384343 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:31 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:31.526021 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42467
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:31.614383 22492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:31.614656 22492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:31.614701 22492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:31.615970 22492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:31.616055 22492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:31.616080 22492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:31.616109 22492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:31.616139 22492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:31.617796 22492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:31.618134 22492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:31.618408 22492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:31.624217 22507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:31.624217 22492 server_base.cc:1034] running on GCE node
W20260809 23:09:31.624228 22509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:31.624228 22506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:31.624768 22492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:31.625191 22492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:31.626425 22492 hybrid_clock.cc:648] HybridClock initialized: now 1786316971626398 us; error 53 us; skew 500 ppm
Aug 09 23:09:31 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316971, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:31.637652 22492 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:31.638229 22492 webserver.cc:460] Webserver started at http://127.8.141.254:42285/ using document root <none> and password file <none>
I20260809 23:09:31.638509 22492 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:31.638583 22492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:31.638749 22492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:31.639870 22492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "26c51623ab17467f9cfb8fcfc51d4f6f"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "13ded05747ca0eb160626cfffb25c2c3"
server_key_iv: "7e23637f783390383e268444a70667e0"
server_key_version: "encryptionkey@0"
I20260809 23:09:31.640254 22492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "26c51623ab17467f9cfb8fcfc51d4f6f"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "13ded05747ca0eb160626cfffb25c2c3"
server_key_iv: "7e23637f783390383e268444a70667e0"
server_key_version: "encryptionkey@0"
I20260809 23:09:31.641903 22492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:31.642879 22516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:31.643148 22492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:31.643275 22492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "26c51623ab17467f9cfb8fcfc51d4f6f"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "13ded05747ca0eb160626cfffb25c2c3"
server_key_iv: "7e23637f783390383e268444a70667e0"
server_key_version: "encryptionkey@0"
I20260809 23:09:31.643360 22492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:31.660943 22492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:31.663705 22492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:31.663934 22492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:31.668895 22492 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42467
I20260809 23:09:31.668998 22605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42467 every 8 connection(s)
I20260809 23:09:31.669755 22492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:31.670727 22606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:31.672778 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 22492
I20260809 23:09:31.672904 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:31.673122 8759 external_mini_cluster.cc:1428] Setting key 39f4fa7d6de0249b4a4846d5d10fe8e9
I20260809 23:09:31.673501 22606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Bootstrap starting.
I20260809 23:09:31.674307 22606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:31.674654 22606 log.cc:824] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:31.675485 22606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: No bootstrap required, opened a new log
I20260809 23:09:31.677407 22606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:31.677561 22606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:31.677584 22606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c51623ab17467f9cfb8fcfc51d4f6f, State: Initialized, Role: FOLLOWER
I20260809 23:09:31.677735 22606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:31.677815 22606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:31.677845 22606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:31.677888 22606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:31.678697 22606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:31.678813 22606 leader_election.cc:302] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c51623ab17467f9cfb8fcfc51d4f6f; no voters:
I20260809 23:09:31.679003 22606 leader_election.cc:288] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:31.679066 22616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:31.679260 22616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 LEADER]: Becoming Leader. State: Replica: 26c51623ab17467f9cfb8fcfc51d4f6f, State: Running, Role: LEADER
I20260809 23:09:31.679316 22606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:31.679371 22616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:31.679744 22617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } } }
I20260809 23:09:31.679832 22617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:31.680018 22618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c51623ab17467f9cfb8fcfc51d4f6f. Latest consensus state: current_term: 1 leader_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } } }
I20260809 23:09:31.680099 22618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:31.680065 22621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:31.680756 22621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:09:31 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:31.682672 22621 catalog_manager.cc:1347] Generated new cluster ID: 9901c0c1f5cb400d931361eec4dd1110
I20260809 23:09:31.682729 22621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:31.685427 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:31.673756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51724 (local address 127.8.141.254:42467)
0809 23:09:31.673976 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:31.673989 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:31.674027 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:09:31.674727 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:31.674736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:31.674801 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:31.674967 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:31.675719 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:31.676745 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:31.677648 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:31.678269 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:31.678399 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:31.681762 (+ 3363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:31.681767 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:31.681768 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:31.681782 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:31.683771 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:31.684157 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:31.684160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:31.684161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:31.684261 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:31.684489 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:31.684490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:31.684491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:31.684753 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:31.684938 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:31.685015 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 23:09:31.685158 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260809 23:09:31.695742 22621 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:31.696329 22621 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:31.708591 22621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Generated new TSK 0
I20260809 23:09:31.708827 22621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:09:31.749619 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42467
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:31.861696 22641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:31.861970 22641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:31.862011 22641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:31.863270 22641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:31.863348 22641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:31.863397 22641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:31.864984 22641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:31.865376 22641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:31.865638 22641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:31.871387 22653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:31.871380 22658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:31.871382 22652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:31.871786 22641 server_base.cc:1034] running on GCE node
I20260809 23:09:31.872107 22641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:31.872468 22641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:31.873662 22641 hybrid_clock.cc:648] HybridClock initialized: now 1786316971873647 us; error 31 us; skew 500 ppm
Aug 09 23:09:31 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316971, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:31.876605 22641 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:09:31.877113 22641 webserver.cc:460] Webserver started at http://127.8.141.193:44829/ using document root <none> and password file <none>
I20260809 23:09:31.877465 22641 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:31.877542 22641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:31.877709 22641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:31.878837 22641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "dfc2df890a1a4c2c8ad2edee6a3936cf"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "417185933f7d991a1f77ac2eb0ce6251"
server_key_iv: "5b037a8bafb4161d00673b702a20b633"
server_key_version: "encryptionkey@0"
I20260809 23:09:31.879230 22641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "dfc2df890a1a4c2c8ad2edee6a3936cf"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "417185933f7d991a1f77ac2eb0ce6251"
server_key_iv: "5b037a8bafb4161d00673b702a20b633"
server_key_version: "encryptionkey@0"
I20260809 23:09:31.880872 22641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:31.881980 22668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:31.882351 22641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:09:31.882572 22641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "dfc2df890a1a4c2c8ad2edee6a3936cf"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "417185933f7d991a1f77ac2eb0ce6251"
server_key_iv: "5b037a8bafb4161d00673b702a20b633"
server_key_version: "encryptionkey@0"
I20260809 23:09:31.882692 22641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:31.910796 22641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:31.913067 22641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:31.913259 22641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:31.913560 22641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:31.913926 22641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:31.913986 22641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:31.914036 22641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:31.914083 22641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:31.920339 22641 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:43497
I20260809 23:09:31.920389 22866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:43497 every 8 connection(s)
I20260809 23:09:31.920737 22641 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:09:31 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316971, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:31.927922 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 22641
I20260809 23:09:31.928046 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:31.928319 8759 external_mini_cluster.cc:1428] Setting key 6b5bafb91557b330355d86049ae4487b
I20260809 23:09:31.931896 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:31.921574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:46177 (local address 127.8.141.254:42467)
0809 23:09:31.921620 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:31.921622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:31.921890 (+ 268us) server_negotiation.cc:407] Connection header received
0809 23:09:31.922474 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:31.922475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:31.922490 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:31.922546 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:31.923621 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:31.924180 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:31.925216 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:31.925492 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:31.925572 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:31.927904 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:31.927906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:31.927906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:31.927914 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:31.930066 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:31.930582 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:31.930584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:31.930584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:31.930660 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:31.930976 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:31.930977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:31.930977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:31.931144 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:31.931229 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:31.931696 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 23:09:31.931788 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:09:31.932206 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:31.921564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:46177)
0809 23:09:31.921748 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:31.921786 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:31.922338 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:31.922627 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:31.922633 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:31.923281 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:31.923502 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:31.923507 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:31.924288 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:31.924289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:31.925117 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:31.925120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:31.925587 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:31.925588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:31.925721 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:31.926282 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:31.926291 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:31.927788 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:31.930180 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:31.930189 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:31.930192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:31.930476 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:31.930762 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:31.930763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:31.930764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:31.930894 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:31.931253 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:31.931264 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:31.931492 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 23:09:31.931650 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 23:09:31.931902 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260809 23:09:31.933307 22867 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42467
I20260809 23:09:31.933527 22867 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:31.933791 22867 heartbeater.cc:511] Master 127.8.141.254:42467 requested a full tablet report, sending...
I20260809 23:09:31.934500 22546 ts_manager.cc:194] Registered new tserver with Master: dfc2df890a1a4c2c8ad2edee6a3936cf (127.8.141.193:43497)
I20260809 23:09:31.935539 22546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:46177
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:09:31.961481 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42467
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:32.068168 22881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:32.068383 22881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:32.068411 22881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:32.069695 22881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:32.069738 22881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:32.069766 22881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:32.071801 22881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:32.072099 22881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:32.072505 22881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:32.078146 22899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:32.078234 22881 server_base.cc:1034] running on GCE node
W20260809 23:09:32.078166 22893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.078146 22896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:32.078614 22881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:32.078922 22881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:32.080122 22881 hybrid_clock.cc:648] HybridClock initialized: now 1786316972080096 us; error 43 us; skew 500 ppm
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316972, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:32.082813 22881 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:09:32.083317 22881 webserver.cc:460] Webserver started at http://127.8.141.194:33215/ using document root <none> and password file <none>
I20260809 23:09:32.083534 22881 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:32.083575 22881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:32.083669 22881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:32.084712 22881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "645110471ba34637b2407a1669a5e654"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "2e81ba5fefd3c758921e8940a70435f3"
server_key_iv: "4eadb8bfde1032fba598957e3b0962af"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.085031 22881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "645110471ba34637b2407a1669a5e654"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "2e81ba5fefd3c758921e8940a70435f3"
server_key_iv: "4eadb8bfde1032fba598957e3b0962af"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.086503 22881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:09:32.087287 22911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.087494 22881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:09:32.087572 22881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "645110471ba34637b2407a1669a5e654"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "2e81ba5fefd3c758921e8940a70435f3"
server_key_iv: "4eadb8bfde1032fba598957e3b0962af"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.087627 22881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:32.099212 22881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:32.101660 22881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:32.101785 22881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:32.101994 22881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:32.102329 22881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:32.102370 22881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.102394 22881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:32.102407 22881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.108762 22881 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:35823
I20260809 23:09:32.108812 23101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:35823 every 8 connection(s)
I20260809 23:09:32.109174 22881 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316972, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:32.117868 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 22881
I20260809 23:09:32.118010 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:32.118217 8759 external_mini_cluster.cc:1428] Setting key 04ab9075c5f9ed72b834a36a8d2e1fd9
I20260809 23:09:32.120349 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.110191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49461 (local address 127.8.141.254:42467)
0809 23:09:32.110275 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.110277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.110507 (+ 230us) server_negotiation.cc:407] Connection header received
0809 23:09:32.111239 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.111241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.111264 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.111351 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.112458 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.112976 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.114055 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.114256 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.114325 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.116747 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:32.116749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:32.116749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:32.116759 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:32.118642 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:32.119242 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:32.119244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:32.119244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:32.119314 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:32.119611 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:32.119612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:32.119613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:32.119744 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:32.119823 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:32.120129 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 23:09:32.120257 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 23:09:32.120644 23104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.110096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:49461)
0809 23:09:32.110339 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.110398 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.111121 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.111483 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.111487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.112135 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.112330 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.112334 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.113069 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.113070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.113919 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.113921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.114360 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.114361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.114480 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.115085 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.115094 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.116627 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:32.118747 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:32.118758 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:32.118760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:32.119130 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:32.119391 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:32.119392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:32.119392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:32.119531 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:32.119813 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:32.119822 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:32.119981 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 23:09:32.120095 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:09:32.120274 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
I20260809 23:09:32.121506 23102 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42467
I20260809 23:09:32.121634 23102 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:32.121872 23102 heartbeater.cc:511] Master 127.8.141.254:42467 requested a full tablet report, sending...
I20260809 23:09:32.122351 22546 ts_manager.cc:194] Registered new tserver with Master: 645110471ba34637b2407a1669a5e654 (127.8.141.194:35823)
I20260809 23:09:32.122763 22546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:49461
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:09:32.149310 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42467
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:32.237102 23108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:32.237332 23108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:32.237396 23108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:32.238685 23108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:32.238737 23108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:32.238767 23108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:32.240409 23108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:32.240703 23108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:32.240976 23108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:32.247470 23120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.247620 23122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.247470 23116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:32.247701 23108 server_base.cc:1034] running on GCE node
I20260809 23:09:32.248060 23108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:32.248785 23108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:32.249974 23108 hybrid_clock.cc:648] HybridClock initialized: now 1786316972249942 us; error 63 us; skew 500 ppm
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316972, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:32.252991 23108 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:09:32.253557 23108 webserver.cc:460] Webserver started at http://127.8.141.195:46801/ using document root <none> and password file <none>
I20260809 23:09:32.253835 23108 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:32.253885 23108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:32.254024 23108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:32.255163 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "6f7be28ebdf748f8bf0c9ea6af19619a"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "4e1088d2c09a8b382f960a575a5d914b"
server_key_iv: "54904a063c11a400e7d306ef86a79829"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.255616 23108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "6f7be28ebdf748f8bf0c9ea6af19619a"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "4e1088d2c09a8b382f960a575a5d914b"
server_key_iv: "54904a063c11a400e7d306ef86a79829"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.257222 23108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:32.258415 23141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.258730 23108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:32.258891 23108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "6f7be28ebdf748f8bf0c9ea6af19619a"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "4e1088d2c09a8b382f960a575a5d914b"
server_key_iv: "54904a063c11a400e7d306ef86a79829"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.258991 23108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:32.274230 23108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:32.277218 23108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:32.277477 23108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:32.277814 23108 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:32.278189 23108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:32.278251 23108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.278298 23108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:32.278350 23108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.287217 23108 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:44471
I20260809 23:09:32.287263 23330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:44471 every 8 connection(s)
I20260809 23:09:32.287640 23108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316972, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:32.296111 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 23108
I20260809 23:09:32.296375 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:32.296603 8759 external_mini_cluster.cc:1428] Setting key 643aa2f8eab0a11205bc207d7077bb61
I20260809 23:09:32.300701 22611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.288756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:42933 (local address 127.8.141.254:42467)
0809 23:09:32.288867 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.288870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.289046 (+ 176us) server_negotiation.cc:407] Connection header received
0809 23:09:32.291237 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.291240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.291256 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.291418 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.292363 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.292906 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.294018 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.294229 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.294328 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.296990 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:32.296992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:32.296992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:32.297003 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:32.299105 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:32.299580 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:32.299582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:32.299582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:32.299651 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:32.299953 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:32.299954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:32.299955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:32.300105 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:32.300170 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:32.300512 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 23:09:32.300601 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 23:09:32.300943 23334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.288604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:42933)
0809 23:09:32.288899 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.288959 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.290744 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.291521 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.291525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.292020 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.292235 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.292240 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.293010 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.293012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.293911 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.293913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.294343 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.294344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.294479 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.295082 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.295092 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.296877 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:32.299224 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:32.299234 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:32.299237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:32.299495 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:32.299731 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:32.299732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:32.299732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:32.299860 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:32.300173 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:32.300182 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:32.300340 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 23:09:32.300474 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 23:09:32.300663 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260809 23:09:32.301609 23332 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42467
I20260809 23:09:32.301723 23332 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:32.301962 23332 heartbeater.cc:511] Master 127.8.141.254:42467 requested a full tablet report, sending...
I20260809 23:09:32.302430 22546 ts_manager.cc:194] Registered new tserver with Master: 6f7be28ebdf748f8bf0c9ea6af19619a (127.8.141.195:44471)
I20260809 23:09:32.302889 22546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:42933
I20260809 23:09:32.309459 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab.
Entry for principal oryx/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab.
WARNING: no policy specified for oryx/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab.
Entry for principal oryx/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab.
WARNING: no policy specified for oryx/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab.
Entry for principal oryx/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab.
I20260809 23:09:32.408990 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 22641
I20260809 23:09:32.429965 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 22881
I20260809 23:09:32.450968 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 23108
I20260809 23:09:32.476300 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 22492
I20260809 23:09:32.494102 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42467
--webserver_interface=127.8.141.254
--webserver_port=42285
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:32.580415 23348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:32.580639 23348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:32.580665 23348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:32.581928 23348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:32.581977 23348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:32.581992 23348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:32.582006 23348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:32.582019 23348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:32.583606 23348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=42285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:32.583928 23348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:32.584188 23348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:32.589453 23361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.589462 23357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.589462 23359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:32.590011 23348 server_base.cc:1034] running on GCE node
I20260809 23:09:32.590390 23348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:32.590747 23348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:32.591960 23348 hybrid_clock.cc:648] HybridClock initialized: now 1786316972591932 us; error 40 us; skew 500 ppm
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316972, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:32.595037 23348 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:32.595593 23348 webserver.cc:460] Webserver started at http://127.8.141.254:42285/ using document root <none> and password file <none>
I20260809 23:09:32.595845 23348 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:32.595949 23348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:32.597502 23348 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:32.598320 23375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.598579 23348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:32.598699 23348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "26c51623ab17467f9cfb8fcfc51d4f6f"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "13ded05747ca0eb160626cfffb25c2c3"
server_key_iv: "7e23637f783390383e268444a70667e0"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.599093 23348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:32.610137 23348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:32.612879 23348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:32.613080 23348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:32.617584 23348 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42467
I20260809 23:09:32.617632 23460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42467 every 8 connection(s)
I20260809 23:09:32.618041 23348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:32.619645 23461 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:09:32.620512 23461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Bootstrap starting.
I20260809 23:09:32.620826 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 23348
I20260809 23:09:32.621152 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:43497
--local_ip_for_outbound_sockets=127.8.141.193
--tserver_master_addrs=127.8.141.254:42467
--webserver_port=44829
--webserver_interface=127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:09:32.630486 23461 log.cc:824] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:32.632360 23461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:09:32.632707 23461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Bootstrap complete.
I20260809 23:09:32.635274 23461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:32.635556 23461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c51623ab17467f9cfb8fcfc51d4f6f, State: Initialized, Role: FOLLOWER
I20260809 23:09:32.635757 23461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:32.635841 23461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:32.635895 23461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:32.635946 23461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:09:32.637069 23461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:32.637202 23461 leader_election.cc:302] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26c51623ab17467f9cfb8fcfc51d4f6f; no voters:
I20260809 23:09:32.637432 23461 leader_election.cc:288] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:09:32.637475 23465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:09:32.637610 23465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [term 2 LEADER]: Becoming Leader. State: Replica: 26c51623ab17467f9cfb8fcfc51d4f6f, State: Running, Role: LEADER
I20260809 23:09:32.637737 23461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:32.637710 23465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } }
I20260809 23:09:32.638041 23465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26c51623ab17467f9cfb8fcfc51d4f6f. Latest consensus state: current_term: 2 leader_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } } }
I20260809 23:09:32.638118 23465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:32.638351 23466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26c51623ab17467f9cfb8fcfc51d4f6f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42467 } } }
I20260809 23:09:32.638404 23470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:32.638442 23466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:32.638974 23470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:32.639132 23470 catalog_manager.cc:1259] Loaded cluster ID: 9901c0c1f5cb400d931361eec4dd1110
I20260809 23:09:32.639176 23470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:32.640296 23470 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:32.640511 23470 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26c51623ab17467f9cfb8fcfc51d4f6f: Loaded TSK: 0
I20260809 23:09:32.640731 23470 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:09:32.715632 23463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:32.715894 23463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:32.715938 23463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:32.717245 23463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:32.717309 23463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:32.717391 23463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:32.718940 23463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.193.keytab
--principal=oryx/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:43497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=44829
--tserver_master_addrs=127.8.141.254:42467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:32.719241 23463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:32.719643 23463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:32.726264 23503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.726231 23510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.726233 23505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:32.726630 23463 server_base.cc:1034] running on GCE node
I20260809 23:09:32.726941 23463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:32.727255 23463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:32.728608 23463 hybrid_clock.cc:648] HybridClock initialized: now 1786316972728596 us; error 38 us; skew 500 ppm
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316972, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:32.731843 23463 init.cc:377] Logged in from keytab as oryx/127.8.141.193@KRBTEST.COM (short username oryx)
I20260809 23:09:32.732474 23463 webserver.cc:460] Webserver started at http://127.8.141.193:44829/ using document root <none> and password file <none>
I20260809 23:09:32.732767 23463 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:32.732863 23463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:32.735021 23463 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:32.735867 23521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.736151 23463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:32.736270 23463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "dfc2df890a1a4c2c8ad2edee6a3936cf"
format_stamp: "Formatted at 2026-08-09 23:09:31 on dist-test-slave-j2vl"
server_key: "417185933f7d991a1f77ac2eb0ce6251"
server_key_iv: "5b037a8bafb4161d00673b702a20b633"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.736817 23463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:32.749250 23463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:32.752342 23463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:32.752528 23463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:32.752825 23463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:32.753229 23463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:32.753280 23463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.753315 23463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:32.753371 23463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.762161 23463 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:43497
I20260809 23:09:32.762250 23706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:43497 every 8 connection(s)
I20260809 23:09:32.762686 23463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:32.767922 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 23463
I20260809 23:09:32.768326 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:35823
--local_ip_for_outbound_sockets=127.8.141.194
--tserver_master_addrs=127.8.141.254:42467
--webserver_port=33215
--webserver_interface=127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:32.775250 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.763555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:43631)
0809 23:09:32.763768 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.763829 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.764354 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.765063 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.765067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.765792 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.765994 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.765999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.767981 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.767982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.769624 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.769629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.770889 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.770890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.771045 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.771738 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.771750 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.775018 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":94,"threads_started":1}
W20260809 23:09:32.775461 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:32.763708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:43631 (local address 127.8.141.254:42467)
0809 23:09:32.763910 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.763919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.763964 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:09:32.764755 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.764763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.764828 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.765003 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.766148 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.767817 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.769838 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.770634 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.770761 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.775227 (+ 4466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:43631: BlockingRecv error: recv got EOF from 127.8.141.193:43631 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
W20260809 23:09:32.775906 23707 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42467 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.141.254:42467: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:32.782322 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.776201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:58321)
0809 23:09:32.776253 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.776264 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.776307 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.776600 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.776601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.776808 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.776877 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.776879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.777919 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.777920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.778727 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.778730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.779306 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.779307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.779404 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.779963 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.779969 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.782246 (+ 2277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 23:09:32.782425 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:32.776250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:58321 (local address 127.8.141.254:42467)
0809 23:09:32.776290 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.776293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.776305 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:32.776490 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.776492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.776507 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.776581 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.776969 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.777817 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.778842 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.779197 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.779287 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.782361 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:58321: BlockingRecv error: recv got EOF from 127.8.141.193:58321 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:32.788398 23710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.782938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:58817)
0809 23:09:32.783056 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.783069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.783128 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.783272 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.783273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.783421 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.783461 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.783463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.784442 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.784443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.785247 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.785249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.785789 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.785790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.785854 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.786348 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.786354 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.788307 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 23:09:32.788610 23707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:09:32.788691 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:32.783008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:58817 (local address 127.8.141.254:42467)
0809 23:09:32.783072 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.783074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.783110 (+ 36us) server_negotiation.cc:407] Connection header received
0809 23:09:32.783178 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.783179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.783190 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.783259 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.783596 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.784354 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.785335 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.785661 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.785726 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.788627 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:58817: BlockingRecv error: recv got EOF from 127.8.141.193:58817 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 23:09:32.866133 23713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:32.866379 23713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:32.866406 23713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:32.867693 23713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:32.867746 23713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:32.867775 23713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:32.869796 23713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.194.keytab
--principal=oryx/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:35823
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=33215
--tserver_master_addrs=127.8.141.254:42467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:32.870090 23713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:32.870368 23713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:09:32.875974 23713 server_base.cc:1034] running on GCE node
W20260809 23:09:32.875954 23721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.876127 23723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:32.875931 23720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:32.876441 23713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:32.876701 23713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:32.877853 23713 hybrid_clock.cc:648] HybridClock initialized: now 1786316972877832 us; error 38 us; skew 500 ppm
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316972, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:32.880568 23713 init.cc:377] Logged in from keytab as oryx/127.8.141.194@KRBTEST.COM (short username oryx)
I20260809 23:09:32.881067 23713 webserver.cc:460] Webserver started at http://127.8.141.194:33215/ using document root <none> and password file <none>
I20260809 23:09:32.881323 23713 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:32.881445 23713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:32.882890 23713 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:32.883538 23733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.883737 23713 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:32.883813 23713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "645110471ba34637b2407a1669a5e654"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "2e81ba5fefd3c758921e8940a70435f3"
server_key_iv: "4eadb8bfde1032fba598957e3b0962af"
server_key_version: "encryptionkey@0"
I20260809 23:09:32.884300 23713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:32.907486 23713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:32.909933 23713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:32.910104 23713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:32.910369 23713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:32.910698 23713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:32.910735 23713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.910787 23713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:32.910815 23713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:32.917313 23713 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:35823
I20260809 23:09:32.917424 23921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:35823 every 8 connection(s)
I20260809 23:09:32.917749 23713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:32.926084 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 23713
I20260809 23:09:32.926427 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:44471
--local_ip_for_outbound_sockets=127.8.141.195
--tserver_master_addrs=127.8.141.254:42467
--webserver_port=46801
--webserver_interface=127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:32.926692 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:32.918631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:53885 (local address 127.8.141.254:42467)
0809 23:09:32.918728 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.918731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.918924 (+ 193us) server_negotiation.cc:407] Connection header received
0809 23:09:32.919523 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.919524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.919544 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.919603 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.920674 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.921235 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.922432 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.922652 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.922750 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.926624 (+ 3874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:53885: BlockingRecv error: recv got EOF from 127.8.141.194:53885 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 23:09:32.926726 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.918529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:53885)
0809 23:09:32.918770 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.918821 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.919394 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.919682 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.919687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.920335 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.920558 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.920564 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.921396 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.921397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.922304 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.922306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.922767 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.922769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.922919 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.923519 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.923529 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.926467 (+ 2938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":112,"threads_started":1}
W20260809 23:09:32.926980 23923 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42467 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.141.254:42467: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:32.932299 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.927290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:49527)
0809 23:09:32.927341 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.927352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.927389 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.927581 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.927582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.927753 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.927817 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.927818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.928768 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.928769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.929321 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.929323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.929799 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.929800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.929878 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.930355 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.930360 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.932187 (+ 1827us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 23:09:32.932346 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:32.927356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49527 (local address 127.8.141.254:42467)
0809 23:09:32.927453 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.927455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.927463 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:32.927489 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.927490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.927499 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.927555 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.927925 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.928664 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.929500 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.929702 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.929799 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.932299 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:49527: BlockingRecv error: recv got EOF from 127.8.141.194:49527 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:32 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:32.937451 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:32.932660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:41675)
0809 23:09:32.932725 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:32.932737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:32.932785 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:32.932938 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:32.932939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:32.933066 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:32.933093 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.933094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.933793 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.933794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.934436 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:32.934440 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.934861 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.934861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:32.934915 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.935486 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:32.935491 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:32.937303 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 23:09:32.937649 23923 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:09:32.937765 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:32.932709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:41675 (local address 127.8.141.254:42467)
0809 23:09:32.932746 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:32.932747 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:32.932771 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:09:32.932843 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:32.932843 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:32.932855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:32.932919 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:32.933170 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.933705 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.934578 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:32.934781 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:32.934861 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:32.937525 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:41675: BlockingRecv error: recv got EOF from 127.8.141.194:41675 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 23:09:33.017621 23930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:33.017853 23930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:33.017880 23930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:33.019266 23930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:33.019322 23930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:33.019358 23930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:33.021085 23930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/krb5kdc/oryx_127.8.141.195.keytab
--principal=oryx/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:44471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=46801
--tserver_master_addrs=127.8.141.254:42467
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:33.021427 23930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:33.021716 23930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:33.028528 23940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:33.028590 23937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:33.028821 23930 server_base.cc:1034] running on GCE node
W20260809 23:09:33.028549 23944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:33.029158 23930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:33.029513 23930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:33.030673 23930 hybrid_clock.cc:648] HybridClock initialized: now 1786316973030658 us; error 31 us; skew 500 ppm
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316973, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:33.033636 23930 init.cc:377] Logged in from keytab as oryx/127.8.141.195@KRBTEST.COM (short username oryx)
I20260809 23:09:33.034171 23930 webserver.cc:460] Webserver started at http://127.8.141.195:46801/ using document root <none> and password file <none>
I20260809 23:09:33.034415 23930 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:33.034477 23930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:33.035964 23930 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:33.036656 23952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:33.036897 23930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:33.036996 23930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "6f7be28ebdf748f8bf0c9ea6af19619a"
format_stamp: "Formatted at 2026-08-09 23:09:32 on dist-test-slave-j2vl"
server_key: "4e1088d2c09a8b382f960a575a5d914b"
server_key_iv: "54904a063c11a400e7d306ef86a79829"
server_key_version: "encryptionkey@0"
I20260809 23:09:33.037415 23930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:33.045902 23930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:33.048275 23930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:33.048444 23930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:33.048695 23930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:33.049029 23930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:33.049086 23930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:33.049131 23930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:33.049160 23930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:33.055802 23930 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:44471
I20260809 23:09:33.055851 24149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:44471 every 8 connection(s)
I20260809 23:09:33.056227 23930 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:33.063162 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 23930
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 23:09:33.065074 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:33.057075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:37279 (local address 127.8.141.254:42467)
0809 23:09:33.057139 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:33.057141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:33.057306 (+ 165us) server_negotiation.cc:407] Connection header received
0809 23:09:33.057819 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:33.057820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:33.057831 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:33.057920 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:33.058985 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.059477 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.060706 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.060909 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.060981 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.065025 (+ 4044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:37279: BlockingRecv error: recv got EOF from 127.8.141.195:37279 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:09:33.065151 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:33.056995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:37279)
0809 23:09:33.057163 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:33.057208 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:33.057722 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:33.058028 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:33.058032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:33.058644 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:33.058850 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.058855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.059587 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.059588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.060581 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.060585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.061017 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.061018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.061139 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.061897 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:33.061906 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:33.064901 (+ 2995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
W20260809 23:09:33.065475 24152 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42467 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.141.254:42467: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:33.070664 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:33.065752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:50551)
0809 23:09:33.065796 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:33.065807 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:33.065847 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:33.065963 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:33.065964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:33.066107 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:33.066164 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.066166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.066804 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.066805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.067314 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.067315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.067641 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.067642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.067724 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.068105 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:33.068110 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:33.070570 (+ 2460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 23:09:33.070734 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:33.065808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:50551 (local address 127.8.141.254:42467)
0809 23:09:33.065853 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:33.065855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:33.065877 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:09:33.065900 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:33.065901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:33.065911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:33.065960 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:33.066254 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.066713 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.067418 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.067586 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.067635 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.070653 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:50551: BlockingRecv error: recv got EOF from 127.8.141.195:50551 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:09:33.072583 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:33.063935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.8.141.254:42467)
0809 23:09:33.064089 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:33.064091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:33.064134 (+ 43us) server_negotiation.cc:407] Connection header received
0809 23:09:33.064158 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:33.064158 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:33.064174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:33.064228 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:33.065330 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.066021 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.066813 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.067034 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.067115 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.068462 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:33.068469 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:33.068469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:33.068485 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:33.070495 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:33.071391 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:33.071395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:33.071396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:33.071515 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:33.071749 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:33.071750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:33.071750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:33.072042 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:33.072219 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:33.072328 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 23:09:33.072454 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:33.076059 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:33.071074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:49759)
0809 23:09:33.071126 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:33.071136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:33.071172 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:33.071406 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:33.071407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:33.071548 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:33.071580 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.071582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.072315 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.072317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.073200 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.073203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.073642 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.073643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.073700 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.074108 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:33.074113 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:33.075963 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 23:09:33.076112 23712 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:33.071199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:49759 (local address 127.8.141.254:42467)
0809 23:09:33.071261 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:33.071263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:33.071270 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:09:33.071290 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:33.071291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:33.071299 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:33.071400 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:33.071664 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.072233 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.073294 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.073552 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.073631 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.076048 (+ 2417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:49759: BlockingRecv error: recv got EOF from 127.8.141.195:49759 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 23:09:33.076258 24152 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:33.794692 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:33.789172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:50633)
0809 23:09:33.789327 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:33.789374 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:33.789426 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:33.789640 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:33.789642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:33.789894 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:33.789960 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.789964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.790732 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.790733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.791302 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.791304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.791668 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.791669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.791716 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.792258 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:33.792263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:33.794616 (+ 2353us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":97,"threads_started":1}
W20260809 23:09:33.794886 24164 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:33.789292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:50633 (local address 127.8.141.254:42467)
0809 23:09:33.789453 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:33.789456 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:33.789464 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:33.789490 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:33.789491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:33.789514 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:33.789620 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:33.790078 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.790633 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.791394 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.791593 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.791650 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.794790 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:50633: BlockingRecv error: recv got EOF from 127.8.141.193:50633 (error 108)
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:33 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:33.943874 24165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:33.938170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:58635)
0809 23:09:33.938360 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:33.938372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:33.938441 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:33.938615 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:33.938616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:33.938798 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:33.938854 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.938862 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.939879 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.939880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.940467 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:33.940469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.940956 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.940957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:33.941012 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.941668 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:33.941679 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:33.943797 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
W20260809 23:09:33.944044 24164 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:33.938279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:58635 (local address 127.8.141.254:42467)
0809 23:09:33.938335 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:33.938338 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:33.938441 (+ 103us) server_negotiation.cc:407] Connection header received
0809 23:09:33.938491 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:33.938492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:33.938505 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:33.938569 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:33.938971 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.939655 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.940593 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:33.940855 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:33.940919 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:33.943963 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:58635: BlockingRecv error: recv got EOF from 127.8.141.194:58635 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 23:09:34 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:34 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:34.082576 24167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:34.076782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:52579)
0809 23:09:34.077037 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:34.077052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:34.077164 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:34.077378 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:34.077379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:34.077571 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:34.077620 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:34.077622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.078333 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.078335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:34.078957 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:34.078959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.079485 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.079486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:34.079565 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:34.080126 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:34.080131 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:34.082499 (+ 2368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":128,"threads_started":1}
W20260809 23:09:34.082715 24164 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:34.076930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:52579 (local address 127.8.141.254:42467)
0809 23:09:34.077042 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:34.077044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:34.077141 (+ 97us) server_negotiation.cc:407] Connection header received
0809 23:09:34.077248 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:34.077249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:34.077264 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:34.077346 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:34.077715 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.078241 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.079050 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.079338 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.079455 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:34.082662 (+ 3207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:52579: BlockingRecv error: recv got EOF from 127.8.141.195:52579 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 23:09:34 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:34 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:34.800761 24168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:34.795371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:46053)
0809 23:09:34.795533 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:34.795591 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:34.795669 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:34.795830 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:34.795831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:34.796020 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:34.796072 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:34.796074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.796758 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.796759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:34.797324 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:34.797326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.797775 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.797776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:34.797872 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:34.798478 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:34.798484 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:34.800683 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260809 23:09:34.800959 24169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:34.795502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:46053 (local address 127.8.141.254:42467)
0809 23:09:34.795668 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:34.795669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:34.795681 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:34.795735 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:34.795735 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:34.795746 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:34.795815 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:34.796142 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.796665 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.797474 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.797680 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.797774 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:34.800839 (+ 3065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:46053: BlockingRecv error: recv got EOF from 127.8.141.193:46053 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
Aug 09 23:09:34 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:34 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:34.949563 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:34.944472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:33525)
0809 23:09:34.944633 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:34.944641 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:34.944718 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:34.944888 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:34.944889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:34.945076 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:34.945125 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:34.945127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.945871 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.945873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:34.946419 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:34.946421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.946800 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.946801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:34.946863 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:34.947400 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:34.947405 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:34.949498 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
W20260809 23:09:34.949805 24169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:34.944530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:33525 (local address 127.8.141.254:42467)
0809 23:09:34.944578 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:34.944580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:34.944706 (+ 126us) server_negotiation.cc:407] Connection header received
0809 23:09:34.944791 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:34.944793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:34.944810 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:34.944866 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:34.945207 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.945786 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.946519 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:34.946718 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:34.946801 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:34.949708 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:33525: BlockingRecv error: recv got EOF from 127.8.141.194:33525 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 23:09:35 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:35 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:35.089118 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:35.083273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:45295)
0809 23:09:35.083460 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:35.083479 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:35.083564 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:35.083756 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:35.083757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:35.083968 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:35.084020 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:35.084023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.084787 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.084788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:35.085401 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:35.085403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.085830 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.085832 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:35.085906 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:35.086472 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:35.086478 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:35.088943 (+ 2465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":122,"threads_started":1}
W20260809 23:09:35.089195 24169 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:35.083485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:45295 (local address 127.8.141.254:42467)
0809 23:09:35.083565 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:35.083568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:35.083580 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:35.083636 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:35.083637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:35.083655 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:35.083743 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:35.084117 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.084691 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.085502 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.085729 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.085818 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:35.089113 (+ 3295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:45295: BlockingRecv error: recv got EOF from 127.8.141.195:45295 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 23:09:35 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:35 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:35.806488 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:35.801458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:56227)
0809 23:09:35.801611 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:35.801624 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:35.801687 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:35.801831 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:35.801832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:35.802009 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:35.802062 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:35.802064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.802816 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.802817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:35.803371 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:35.803374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.803728 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.803729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:35.803776 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:35.804262 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:35.804267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:35.806423 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
W20260809 23:09:35.806545 24174 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:35.801553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:56227 (local address 127.8.141.254:42467)
0809 23:09:35.801693 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:35.801695 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:35.801705 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:09:35.801731 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:35.801731 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:35.801742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:35.801811 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:35.802131 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.802678 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.803457 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.803652 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.803708 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:35.806491 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:56227: BlockingRecv error: recv got EOF from 127.8.141.193:56227 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
Aug 09 23:09:35 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:35 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:35.955019 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:35.950222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:37401)
0809 23:09:35.950366 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:35.950381 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:35.950494 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:35.950686 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:35.950687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:35.950934 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:35.951003 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:35.951006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.951734 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.951735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:35.952261 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:35.952262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.952607 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.952608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:35.952652 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:35.953120 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:35.953124 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:35.954958 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
W20260809 23:09:35.955247 24174 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:35.950310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:37401 (local address 127.8.141.254:42467)
0809 23:09:35.950366 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:35.950368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:35.950476 (+ 108us) server_negotiation.cc:407] Connection header received
0809 23:09:35.950584 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:35.950585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:35.950599 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:35.950661 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:35.951106 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.951656 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.952342 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:35.952532 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:35.952589 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:35.955193 (+ 2604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:37401: BlockingRecv error: recv got EOF from 127.8.141.194:37401 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 23:09:36 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:36 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:36.094404 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:36.089736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:58471)
0809 23:09:36.089932 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:36.089942 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:36.089986 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:36.090134 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:36.090135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:36.090302 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:36.090355 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:36.090356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.091035 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.091036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:36.091589 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:36.091591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.091943 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.091944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:36.091998 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:36.092465 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:36.092470 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:36.094342 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":62,"threads_started":1}
W20260809 23:09:36.094558 24174 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:36.089937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:58471 (local address 127.8.141.254:42467)
0809 23:09:36.090007 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:36.090009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:36.090017 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:36.090041 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:36.090042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:36.090053 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:36.090120 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:36.090437 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.090933 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.091662 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.091841 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.091891 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:36.094458 (+ 2567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:58471: BlockingRecv error: recv got EOF from 127.8.141.195:58471 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 23:09:36 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:36 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:36.817569 24181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:36.807095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:49809)
0809 23:09:36.807260 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:36.807271 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:36.807322 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:36.807480 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:36.807481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:36.807658 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:36.807711 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:36.807713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.808405 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.808406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:36.808958 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:36.808960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.809437 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.809438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:36.809496 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:36.815293 (+ 5797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:36.815299 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:36.817507 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
W20260809 23:09:36.817780 24182 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:36.807162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:49809 (local address 127.8.141.254:42467)
0809 23:09:36.807342 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:36.807344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:36.807353 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:36.807377 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:36.807377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:36.807387 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:36.807457 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:36.807805 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.808320 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.809054 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.809328 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.809431 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:36.817655 (+ 8224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:49809: BlockingRecv error: recv got EOF from 127.8.141.193:49809 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":103,"threads_started":1}
Aug 09 23:09:36 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:36 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:36.960565 24183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:36.955599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:47023)
0809 23:09:36.955767 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:36.955780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:36.955855 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:36.956028 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:36.956030 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:36.956211 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:36.956263 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:36.956264 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.956952 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.956953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:36.957607 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:36.957609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.957987 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.957988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:36.958042 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:36.958619 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:36.958624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:36.960503 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
W20260809 23:09:36.960757 24182 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:36.955724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:47023 (local address 127.8.141.254:42467)
0809 23:09:36.955781 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:36.955783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:36.955856 (+ 73us) server_negotiation.cc:407] Connection header received
0809 23:09:36.955907 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:36.955908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:36.955920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:36.955991 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:36.956354 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.956856 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.957695 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:36.957897 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:36.957956 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:36.960658 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:47023: BlockingRecv error: recv got EOF from 127.8.141.194:47023 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 23:09:37 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:37 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:37.099922 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:37.094968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:53531)
0809 23:09:37.095162 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:37.095174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:37.095233 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:37.095391 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:37.095392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:37.095578 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:37.095631 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:37.095633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.096313 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.096314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:37.096876 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:37.096877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.097245 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.097246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:37.097300 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:37.097844 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:37.097849 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:37.099828 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":99,"threads_started":1}
W20260809 23:09:37.100090 24182 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:37.095006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:53531 (local address 127.8.141.254:42467)
0809 23:09:37.095089 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:37.095091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:37.095235 (+ 144us) server_negotiation.cc:407] Connection header received
0809 23:09:37.095288 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:37.095289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:37.095302 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:37.095346 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:37.095714 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.096231 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.096963 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.097151 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.097210 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:37.100021 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:53531: BlockingRecv error: recv got EOF from 127.8.141.195:53531 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 23:09:37 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:37 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:37.823236 24185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:37.818110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:44251)
0809 23:09:37.818273 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:37.818288 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:37.818399 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:37.818588 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:37.818589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:37.818778 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:37.818835 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:37.818837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.819532 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.819533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:37.820081 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:37.820083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.820459 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.820460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:37.820503 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:37.821029 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:37.821034 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:37.823173 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":73,"threads_started":1}
W20260809 23:09:37.823468 24186 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:37.818186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:44251 (local address 127.8.141.254:42467)
0809 23:09:37.818310 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:37.818313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:37.818396 (+ 83us) server_negotiation.cc:407] Connection header received
0809 23:09:37.818462 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:37.818463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:37.818494 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:37.818568 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:37.818925 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.819439 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.820180 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.820386 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.820457 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:37.823371 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:44251: BlockingRecv error: recv got EOF from 127.8.141.193:44251 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":51,"threads_started":1}
Aug 09 23:09:37 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:37 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:37.966689 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:37.961335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:50527)
0809 23:09:37.961555 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:37.961566 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:37.961635 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:37.961790 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:37.961791 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:37.961981 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:37.962033 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:37.962035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.962813 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.962814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:37.963424 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:37.963426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.963882 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.963883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:37.963944 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:37.964476 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:37.964482 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:37.966620 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":113,"threads_started":1}
W20260809 23:09:37.966905 24186 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:37.961399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:50527 (local address 127.8.141.254:42467)
0809 23:09:37.961518 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:37.961521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:37.961639 (+ 118us) server_negotiation.cc:407] Connection header received
0809 23:09:37.961702 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:37.961703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:37.961716 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:37.961761 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:37.962137 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.962696 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.963528 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:37.963809 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:37.963883 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:37.966784 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:50527: BlockingRecv error: recv got EOF from 127.8.141.194:50527 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 23:09:38 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:38 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:38.105765 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:38.100570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:60071)
0809 23:09:38.100806 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:38.100818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:38.100915 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:38.101082 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:38.101083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:38.101360 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:38.101427 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:38.101429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.102108 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.102109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:38.102707 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:38.102709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.103090 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.103091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:38.103150 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:38.103634 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:38.103639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:38.105704 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":129,"threads_started":1}
W20260809 23:09:38.105943 24186 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:38.100675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:60071 (local address 127.8.141.254:42467)
0809 23:09:38.100785 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:38.100787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:38.100893 (+ 106us) server_negotiation.cc:407] Connection header received
0809 23:09:38.100981 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:38.100982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:38.101000 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:38.101073 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:38.101522 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.102022 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.102791 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.102982 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.103074 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:38.105830 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:60071: BlockingRecv error: recv got EOF from 127.8.141.195:60071 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 23:09:38 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:38 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:38.829282 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:38.823861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:39689)
0809 23:09:38.824052 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:38.824068 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:38.824141 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:38.824316 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:38.824317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:38.824506 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:38.824558 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:38.824560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.825326 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.825327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:38.825988 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:38.825990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.826414 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.826415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:38.826474 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:38.826980 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:38.826985 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:38.829212 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
W20260809 23:09:38.829602 24191 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:38.823986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:39689 (local address 127.8.141.254:42467)
0809 23:09:38.824137 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:38.824139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:38.824149 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:09:38.824192 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:38.824192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:38.824203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:38.824263 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:38.824676 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.825207 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.826089 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.826296 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.826363 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:38.829441 (+ 3078us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:39689: BlockingRecv error: recv got EOF from 127.8.141.193:39689 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
Aug 09 23:09:38 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:38 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:38.972671 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:38.967287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:47245)
0809 23:09:38.967527 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:38.967543 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:38.967630 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:38.967778 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:38.967779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:38.968014 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:38.968065 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:38.968067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.968836 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.968837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:38.969451 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:38.969454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.969872 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.969873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:38.969940 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:38.970431 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:38.970437 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:38.972559 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":110,"threads_started":1}
W20260809 23:09:38.972724 24191 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:38.967427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:47245 (local address 127.8.141.254:42467)
0809 23:09:38.967533 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:38.967536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:38.967625 (+ 89us) server_negotiation.cc:407] Connection header received
0809 23:09:38.967706 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:38.967707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:38.967720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:38.967773 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:38.968160 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.968738 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.969549 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:38.969782 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:38.969873 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:38.972651 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:47245: BlockingRecv error: recv got EOF from 127.8.141.194:47245 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 23:09:39 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:39 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:39.111968 24194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:39.106402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:57957)
0809 23:09:39.106701 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:39.106717 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:39.106851 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:39.107039 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:39.107040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:39.107239 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:39.107291 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:39.107293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.107999 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.108001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:39.108531 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:39.108532 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.108897 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.108898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:39.108960 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:39.109554 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:39.109560 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:39.111844 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":104,"threads_started":1}
W20260809 23:09:39.112004 24191 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:39.106518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:57957 (local address 127.8.141.254:42467)
0809 23:09:39.106706 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:39.106709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:39.106826 (+ 117us) server_negotiation.cc:407] Connection header received
0809 23:09:39.106942 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:39.106943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:39.106961 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:39.107018 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:39.107385 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.107890 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.108622 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.108818 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.108897 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:39.111942 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:57957: BlockingRecv error: recv got EOF from 127.8.141.195:57957 (error 108)
Metrics: {"server-negotiator.queue_time_us":156}
Aug 09 23:09:39 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:39 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:39.835332 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:39.830077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:34723)
0809 23:09:39.830298 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:39.830311 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:39.830377 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:39.830534 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:39.830534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:39.830708 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:39.830768 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:39.830770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.831550 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.831551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:39.832325 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:39.832329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.832698 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.832699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:39.832746 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:39.833223 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:39.833228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:39.835269 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":120,"threads_started":1}
W20260809 23:09:39.835516 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:39.830221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:34723 (local address 127.8.141.254:42467)
0809 23:09:39.830376 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:39.830378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:39.830389 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:39.830440 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:39.830441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:39.830451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:39.830512 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:39.830862 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.831436 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.832412 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.832628 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.832682 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:39.835404 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:34723: BlockingRecv error: recv got EOF from 127.8.141.193:34723 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
Aug 09 23:09:39 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:39 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:39.978760 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:39.973449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:44123)
0809 23:09:39.973671 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:39.973681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:39.973775 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:39.973947 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:39.973948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:39.974166 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:39.974217 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:39.974220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.974953 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.974961 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:39.975497 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:39.975499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.975888 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.975889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:39.975951 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:39.976484 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:39.976489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:39.978698 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":145,"threads_started":1}
W20260809 23:09:39.978904 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:39.973506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:44123 (local address 127.8.141.254:42467)
0809 23:09:39.973590 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:39.973593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:39.973757 (+ 164us) server_negotiation.cc:407] Connection header received
0809 23:09:39.973861 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:39.973862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:39.973877 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:39.973928 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:39.974306 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.974850 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.975589 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:39.975796 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:39.975886 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:39.978809 (+ 2923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:44123: BlockingRecv error: recv got EOF from 127.8.141.194:44123 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 23:09:40 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:40 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:40.117271 24202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:40.112546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:41373)
0809 23:09:40.112724 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:40.112733 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:40.112775 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:40.112906 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:40.112907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:40.113077 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:40.113131 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:40.113133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.113830 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.113831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:40.114372 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:40.114374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.114736 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.114737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:40.114819 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:40.115299 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:40.115305 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:40.117208 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
W20260809 23:09:40.117520 24198 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:40.112717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:41373 (local address 127.8.141.254:42467)
0809 23:09:40.112785 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:40.112787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:40.112794 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:09:40.112816 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:40.112816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:40.112830 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:40.112884 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:40.113203 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.113759 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.114448 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.114627 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.114683 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:40.117443 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:41373: BlockingRecv error: recv got EOF from 127.8.141.195:41373 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 23:09:40 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:40 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:40.842075 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:40.835972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:37311)
0809 23:09:40.836236 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:40.836253 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:40.836326 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:40.836674 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:40.836676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:40.836895 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:40.836949 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:40.836951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.838010 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.838014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:40.838565 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:40.838567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.839156 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.839158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:40.839317 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:40.839821 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:40.839827 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:40.841972 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":67,"threads_started":1}
W20260809 23:09:40.842289 24207 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:40.836320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:37311 (local address 127.8.141.254:42467)
0809 23:09:40.836468 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:40.836470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:40.836479 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:40.836502 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:40.836503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:40.836514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:40.836590 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:40.837057 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.837868 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.838687 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.839046 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.839113 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:40.842197 (+ 3084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:37311: BlockingRecv error: recv got EOF from 127.8.141.193:37311 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
Aug 09 23:09:40 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:40 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:40.985400 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:40.979447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:53945)
0809 23:09:40.979583 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:40.979601 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:40.979654 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:40.979809 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:40.979809 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:40.979984 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:40.980041 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:40.980042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.980934 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.980935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:40.982198 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:40.982203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.982637 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.982639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:40.982688 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:40.983211 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:40.983216 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:40.985265 (+ 2049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
W20260809 23:09:40.985491 24207 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:40.979548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:53945 (local address 127.8.141.254:42467)
0809 23:09:40.979658 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:40.979661 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:40.979678 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:09:40.979725 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:40.979726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:40.979737 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:40.979802 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:40.980110 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.980798 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.982338 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:40.982542 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:40.982610 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:40.985422 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:53945: BlockingRecv error: recv got EOF from 127.8.141.194:53945 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 23:09:41 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:41 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:41.122838 24210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:41.117858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:39699)
0809 23:09:41.117998 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:41.118016 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:41.118100 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:41.118252 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:41.118253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:41.118458 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:41.118509 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:41.118511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.119210 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.119211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:41.119752 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:41.119754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.120131 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.120132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:41.120194 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:41.120744 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:41.120750 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:41.122759 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
W20260809 23:09:41.123116 24207 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:41.117886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:39699 (local address 127.8.141.254:42467)
0809 23:09:41.117956 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:41.117959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:41.118079 (+ 120us) server_negotiation.cc:407] Connection header received
0809 23:09:41.118158 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:41.118159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:41.118179 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:41.118237 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:41.118596 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.119117 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.119843 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.120052 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.120132 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:41.122917 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:39699: BlockingRecv error: recv got EOF from 127.8.141.195:39699 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 23:09:41 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:41 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:41.852568 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:41.842689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:53843)
0809 23:09:41.842841 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:41.842853 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:41.842930 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:41.843165 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:41.843166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:41.843343 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:41.843398 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:41.843399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.844129 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.844130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:41.844663 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:41.844665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.845089 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.845090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:41.845148 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:41.850127 (+ 4979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:41.850136 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:41.852500 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
W20260809 23:09:41.852793 24212 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:41.842767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:53843 (local address 127.8.141.254:42467)
0809 23:09:41.843023 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:41.843025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:41.843035 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:09:41.843065 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:41.843065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:41.843076 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:41.843150 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:41.843495 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.844033 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.844755 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.844992 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.845054 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:41.852699 (+ 7645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:53843: BlockingRecv error: recv got EOF from 127.8.141.193:53843 (error 108)
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":194,"threads_started":1}
Aug 09 23:09:41 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:41 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:41.993602 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:41.985958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:47031)
0809 23:09:41.986131 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:41.986142 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:41.986200 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:41.986329 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:41.986330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:41.986514 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:41.986562 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:41.986564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.987310 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.987311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:41.987896 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:41.987898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.988264 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.988266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:41.988329 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:41.991511 (+ 3182us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:41.991517 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:41.993529 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
W20260809 23:09:41.993851 24212 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:41.986084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:47031 (local address 127.8.141.254:42467)
0809 23:09:41.986147 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:41.986149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:41.986185 (+ 36us) server_negotiation.cc:407] Connection header received
0809 23:09:41.986250 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:41.986251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:41.986263 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:41.986323 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:41.986657 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.987169 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.987991 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:41.988189 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:41.988241 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:41.993753 (+ 5512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:47031: BlockingRecv error: recv got EOF from 127.8.141.194:47031 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 23:09:42 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:42 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:42.128553 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:42.123463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:35319)
0809 23:09:42.123609 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:42.123619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:42.123689 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:42.123827 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:42.123832 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:42.124017 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:42.124069 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:42.124070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.124817 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.124818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:42.125456 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:42.125459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.125874 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.125875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:42.125929 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:42.126437 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:42.126443 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:42.128448 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
W20260809 23:09:42.128607 24212 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:42.123573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:35319 (local address 127.8.141.254:42467)
0809 23:09:42.123622 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:42.123626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:42.123673 (+ 47us) server_negotiation.cc:407] Connection header received
0809 23:09:42.123736 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:42.123737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:42.123750 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:42.123838 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:42.124158 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.124704 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.125566 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.125770 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.125839 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:42.128538 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:35319: BlockingRecv error: recv got EOF from 127.8.141.195:35319 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 23:09:42 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:42 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:42.859025 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:42.853160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:49149)
0809 23:09:42.853302 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:42.853315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:42.853424 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:42.853605 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:42.853606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:42.853784 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:42.853838 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:42.853839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.854572 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.854573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:42.855292 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:42.855296 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.855832 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.855833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:42.855889 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:42.856477 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:42.856483 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:42.858946 (+ 2463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
W20260809 23:09:42.859163 24218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:42.853309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:49149 (local address 127.8.141.254:42467)
0809 23:09:42.853475 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:42.853477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:42.853485 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:09:42.853507 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:42.853508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:42.853518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:42.853585 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:42.853948 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.854472 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.855413 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.855717 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.855791 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:42.859077 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:49149: BlockingRecv error: recv got EOF from 127.8.141.193:49149 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
Aug 09 23:09:42 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:42 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:42.999571 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:42.994204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:55939)
0809 23:09:42.994360 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:42.994374 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:42.994451 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:42.994622 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:42.994623 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:42.994847 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:42.994899 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:42.994901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.995634 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.995635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:42.996220 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:42.996222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.996618 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.996620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:42.996687 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:42.997194 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:42.997198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:42.999505 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
W20260809 23:09:42.999747 24218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:42.994346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:55939 (local address 127.8.141.254:42467)
0809 23:09:42.994443 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:42.994445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:42.994458 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:09:42.994524 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:42.994525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:42.994538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:42.994612 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:42.995002 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.995539 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.996323 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:42.996528 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:42.996606 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:42.999660 (+ 3054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:55939: BlockingRecv error: recv got EOF from 127.8.141.194:55939 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 23:09:43 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:43 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:43.135046 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:43.129254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:39233)
0809 23:09:43.129452 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:43.129466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:43.129538 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:43.129833 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:43.129834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:43.130049 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:43.130100 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:43.130102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.130811 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.130813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:43.131368 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:43.131370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.131753 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.131754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:43.131821 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:43.132389 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:43.132395 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:43.134968 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":156,"threads_started":1}
W20260809 23:09:43.135406 24218 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:43.129560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:39233 (local address 127.8.141.254:42467)
0809 23:09:43.129672 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:43.129674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:43.129683 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:43.129714 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:43.129715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:43.129727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:43.129821 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:43.130191 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.130711 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.131460 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.131662 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.131738 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:43.135321 (+ 3583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:39233: BlockingRecv error: recv got EOF from 127.8.141.195:39233 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 23:09:43 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:43 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:43.865409 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:43.859741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:40815)
0809 23:09:43.859935 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:43.859951 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:43.860018 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:43.860279 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:43.860280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:43.860488 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:43.860538 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:43.860540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.861320 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.861321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:43.862090 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:43.862093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.862557 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.862558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:43.862624 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:43.863255 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:43.863261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:43.865264 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
W20260809 23:09:43.865463 24227 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:43.859957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:40815 (local address 127.8.141.254:42467)
0809 23:09:43.860087 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:43.860089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:43.860098 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:09:43.860126 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:43.860126 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:43.860137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:43.860199 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:43.860723 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.861249 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.862208 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:43.862425 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:43.862493 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:43.865384 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:40815: BlockingRecv error: recv got EOF from 127.8.141.193:40815 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
Aug 09 23:09:44 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:44 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:44.008571 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:44.000159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:50363)
0809 23:09:44.000364 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:44.000375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:44.000441 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:44.000631 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:44.000632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:44.000834 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:44.000888 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:44.000889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.001812 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.001813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:44.002877 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:44.002882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.003332 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.003333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:44.003389 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:44.004179 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:44.004185 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:44.008438 (+ 4253us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
W20260809 23:09:44.008673 24227 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:44.000306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:50363 (local address 127.8.141.254:42467)
0809 23:09:44.000421 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:44.000424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:44.000439 (+ 15us) server_negotiation.cc:407] Connection header received
0809 23:09:44.000497 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:44.000498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:44.000511 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:44.000581 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:44.000994 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.001692 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.003003 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.003228 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.003297 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:44.008596 (+ 5299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:50363: BlockingRecv error: recv got EOF from 127.8.141.194:50363 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 23:09:44 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:44 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:44.141224 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:44.135760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:50821)
0809 23:09:44.135964 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:44.135975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:44.136066 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:44.136231 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:44.136232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:44.136444 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:44.136493 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:44.136495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.137180 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.137182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:44.137813 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:44.137816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.138178 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.138179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:44.138241 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:44.138838 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:44.138843 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:44.141086 (+ 2243us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":112,"threads_started":1}
W20260809 23:09:44.141269 24227 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:44.135876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:50821 (local address 127.8.141.254:42467)
0809 23:09:44.135950 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:44.135953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:44.136046 (+ 93us) server_negotiation.cc:407] Connection header received
0809 23:09:44.136130 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:44.136131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:44.136146 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:44.136223 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:44.136576 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.137085 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.137902 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.138094 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.138172 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:44.141192 (+ 3020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:50821: BlockingRecv error: recv got EOF from 127.8.141.195:50821 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 23:09:44 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:44 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:44.871899 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:44.866318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:53965)
0809 23:09:44.866488 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:44.866503 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:44.866563 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:44.866785 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:44.866786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:44.866977 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:44.867030 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:44.867032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.867751 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.867752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:44.868320 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:44.868322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.868875 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.868877 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:44.868930 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:44.869565 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:44.869572 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:44.871822 (+ 2250us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
W20260809 23:09:44.872103 24231 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:44.866445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:53965 (local address 127.8.141.254:42467)
0809 23:09:44.866601 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:44.866603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:44.866615 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:44.866652 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:44.866653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:44.866664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:44.866740 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:44.867111 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.867644 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.868445 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:44.868662 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:44.868846 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:44.872008 (+ 3162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:53965: BlockingRecv error: recv got EOF from 127.8.141.193:53965 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
Aug 09 23:09:45 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:45 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:45.015515 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:45.009496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:55077)
0809 23:09:45.009631 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:45.009651 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:45.009701 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:45.010291 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:45.010292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:45.010486 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:45.010539 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:45.010541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.011270 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.011271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:45.011898 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:45.011901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.012693 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.012694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:45.012746 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:45.013392 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:45.013397 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:45.015439 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
W20260809 23:09:45.015609 24231 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:45.009703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:55077 (local address 127.8.141.254:42467)
0809 23:09:45.009784 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:45.009787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:45.009801 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:09:45.009824 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:45.009825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:45.009836 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:45.010263 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:45.010631 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.011165 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.012079 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.012570 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.012642 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:45.015556 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:55077: BlockingRecv error: recv got EOF from 127.8.141.194:55077 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 23:09:45 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:45 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:45.147801 24234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:45.141858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:42611)
0809 23:09:45.142028 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:45.142046 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:45.142109 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:45.142295 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:45.142296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:45.142481 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:45.142531 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:45.142533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.143293 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.143294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:45.143846 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:45.143848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.144313 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.144315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:45.144377 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:45.145194 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:45.145201 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:45.147687 (+ 2486us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
W20260809 23:09:45.147871 24231 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:45.141951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:42611 (local address 127.8.141.254:42467)
0809 23:09:45.142023 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:45.142025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:45.142111 (+ 86us) server_negotiation.cc:407] Connection header received
0809 23:09:45.142156 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:45.142157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:45.142170 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:45.142276 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:45.142669 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.143187 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.144000 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.144189 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.144275 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:45.147788 (+ 3513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:42611: BlockingRecv error: recv got EOF from 127.8.141.195:42611 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 23:09:45 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:45 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:45.877590 24236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:45.872596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:43283)
0809 23:09:45.872769 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:45.872782 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:45.872839 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:45.872967 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:45.872968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:45.873149 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:45.873207 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:45.873208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.873960 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.873961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:45.874508 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:45.874511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.874904 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.874905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:45.874948 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:45.875548 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:45.875554 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:45.877472 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":85,"threads_started":1}
W20260809 23:09:45.877643 24235 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:45.872604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:43283 (local address 127.8.141.254:42467)
0809 23:09:45.872817 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:45.872844 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:45.872855 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:45.872880 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:45.872881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:45.872893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:45.872956 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:45.873289 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.873875 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.874598 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:45.874808 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:45.874890 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:45.877564 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:43283: BlockingRecv error: recv got EOF from 127.8.141.193:43283 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
Aug 09 23:09:46 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:46 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:46.022150 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:46.016378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:33189)
0809 23:09:46.016565 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:46.016604 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:46.016699 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:46.016879 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:46.016880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:46.017145 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:46.017230 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:46.017232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.018030 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.018031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:46.018636 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:46.018638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.019091 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.019092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:46.019147 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:46.019774 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:46.019779 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:46.022051 (+ 2272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":78,"threads_started":1}
W20260809 23:09:46.022383 24235 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:46.016614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:33189 (local address 127.8.141.254:42467)
0809 23:09:46.016689 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:46.016692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:46.016706 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:09:46.016774 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:46.016775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:46.016791 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:46.016853 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:46.017325 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.017921 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.018749 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.018980 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.019050 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:46.022159 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:33189: BlockingRecv error: recv got EOF from 127.8.141.194:33189 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 23:09:46 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:46 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:46.153838 24239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:46.148463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:33719)
0809 23:09:46.148797 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:46.148815 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:46.148921 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:46.149114 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:46.149116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:46.149320 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:46.149422 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:46.149424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.150136 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.150138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:46.150706 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:46.150709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.151094 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.151095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:46.151154 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:46.151702 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:46.151707 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:46.153760 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":118,"threads_started":1}
W20260809 23:09:46.154208 24235 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:46.148572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:33719 (local address 127.8.141.254:42467)
0809 23:09:46.148774 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:46.148777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:46.148896 (+ 119us) server_negotiation.cc:407] Connection header received
0809 23:09:46.149001 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:46.149002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:46.149017 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:46.149096 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:46.149507 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.150040 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.150810 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.151014 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.151095 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:46.154084 (+ 2989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:33719: BlockingRecv error: recv got EOF from 127.8.141.195:33719 (error 108)
Metrics: {"server-negotiator.queue_time_us":166}
Aug 09 23:09:46 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:46 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:46.887425 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:46.878251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:40739)
0809 23:09:46.878566 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:46.878579 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:46.878648 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:46.878880 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:46.878881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:46.879070 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:46.879120 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:46.879122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.879871 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.879872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:46.880432 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:46.880435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.880824 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.880825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:46.880875 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:46.884974 (+ 4099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:46.884981 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:46.887329 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":205,"threads_started":1}
W20260809 23:09:46.887578 24243 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:46.878462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:40739 (local address 127.8.141.254:42467)
0809 23:09:46.878602 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:46.878604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:46.878631 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:09:46.878743 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:46.878743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:46.878755 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:46.878819 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:46.879244 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.879769 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.880548 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:46.880749 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:46.880817 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:46.887518 (+ 6701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:40739: BlockingRecv error: recv got EOF from 127.8.141.193:40739 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
Aug 09 23:09:47 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:47 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:47.030301 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:47.022860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:59595)
0809 23:09:47.023059 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:47.023072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:47.023140 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:47.023398 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:47.023399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:47.023590 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:47.023642 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:47.023644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.024648 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.024650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:47.025801 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:47.025806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.026645 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.026645 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:47.026700 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:47.027738 (+ 1038us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:47.027743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:47.030195 (+ 2452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":70,"threads_started":1}
W20260809 23:09:47.030614 24243 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:47.023106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:59595 (local address 127.8.141.254:42467)
0809 23:09:47.023199 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:47.023201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:47.023212 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:47.023245 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:47.023246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:47.023258 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:47.023326 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:47.023802 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.024517 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.026068 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.026528 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.026612 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:47.030553 (+ 3941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:59595: BlockingRecv error: recv got EOF from 127.8.141.194:59595 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 23:09:47 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:47 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.195@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:47.160272 24247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:47.154479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.195:44139)
0809 23:09:47.154779 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:47.154790 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:47.154846 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:47.155104 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:47.155105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:47.155416 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:47.155487 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:47.155490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.156332 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.156333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:47.156983 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:47.156986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.157518 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.157519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:47.157571 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:47.158165 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:47.158171 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:47.160207 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":210,"threads_started":1}
W20260809 23:09:47.160590 24243 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:47.154738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:44139 (local address 127.8.141.254:42467)
0809 23:09:47.154822 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:47.154823 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:47.154835 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:47.154972 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:47.154974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:47.154988 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:47.155056 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:47.155704 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.156238 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.157118 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.157331 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.157453 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:47.160521 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.195:44139: BlockingRecv error: recv got EOF from 127.8.141.195:44139 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 23:09:47 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:47 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.193@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:47.893817 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:47.888577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.193:46373)
0809 23:09:47.888730 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:47.888750 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:47.888811 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:47.888969 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:47.888969 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:47.889150 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:47.889206 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:47.889208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.889976 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.889978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:47.890552 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:47.890555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.890983 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.890985 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:47.891039 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:47.891551 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:47.891555 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:47.893698 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
W20260809 23:09:47.893869 24250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:47.888675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:46373 (local address 127.8.141.254:42467)
0809 23:09:47.888834 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:47.888836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:47.888847 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:09:47.888871 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:47.888872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:47.888882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:47.888950 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:47.889297 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.889852 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.890655 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:47.890881 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:47.890948 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:47.893792 (+ 2844us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.193:46373: BlockingRecv error: recv got EOF from 127.8.141.193:46373 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
Aug 09 23:09:48 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 23:09:48 dist-test-slave-j2vl krb5kdc[22475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.141.194@KRBTEST.COM for oryx/127.8.141.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 23:09:48.035820 24253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.030976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42467 (local address 127.8.141.194:49339)
0809 23:09:48.031122 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:48.031133 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:48.031184 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:48.031369 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:48.031370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:48.031547 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:48.031599 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.031600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.032243 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.032244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.032789 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.032790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.033143 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.033144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.033196 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.033779 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:48.033784 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:48.035754 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.141.254:42467: Server oryx/127.8.141.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260809 23:09:48.036043 24250 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:48.031120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49339 (local address 127.8.141.254:42467)
0809 23:09:48.031179 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:48.031180 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:48.031194 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:09:48.031246 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:48.031247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:48.031258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:48.031316 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:48.031676 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.032166 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.032862 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.033051 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.033104 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.035980 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.141.194:49339: BlockingRecv error: recv got EOF from 127.8.141.194:49339 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:48.071908 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 23463
I20260809 23:09:48.092864 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 23713
I20260809 23:09:48.118432 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 23930
I20260809 23:09:48.135850 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 23348
2026-08-09T23:09:48Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18055 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T23:09:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:48Z Disabled control of system clock
I20260809 23:09:48.179237 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36187
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:36187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 23:09:48.267704 24260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:48.267946 24260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:48.267967 24260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:48.269268 24260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:48.269335 24260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:48.269368 24260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:48.269382 24260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:48.269394 24260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:48.270906 24260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:36187
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:48.271224 24260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:48.271581 24260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:48.277998 24271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:48.278053 24260 server_base.cc:1034] running on GCE node
W20260809 23:09:48.277999 24269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.277998 24267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:48.278683 24260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:48.279014 24260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:48.280215 24260 hybrid_clock.cc:648] HybridClock initialized: now 1786316988280179 us; error 59 us; skew 500 ppm
I20260809 23:09:48.282119 24260 webserver.cc:460] Webserver started at http://127.8.141.254:34861/ using document root <none> and password file <none>
I20260809 23:09:48.282426 24260 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:48.282510 24260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:48.282676 24260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:48.283812 24260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "a1909b8b43d249428e2bf988b735fa5c"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "c504893701f235c0c1d37e94ce2f28c5"
server_key_iv: "16d657076e8710bcdc0fcd28026e4bde"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.284224 24260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "a1909b8b43d249428e2bf988b735fa5c"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "c504893701f235c0c1d37e94ce2f28c5"
server_key_iv: "16d657076e8710bcdc0fcd28026e4bde"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.285708 24260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:48.286636 24282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.286875 24260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:48.287001 24260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "a1909b8b43d249428e2bf988b735fa5c"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "c504893701f235c0c1d37e94ce2f28c5"
server_key_iv: "16d657076e8710bcdc0fcd28026e4bde"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.287083 24260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:48.303205 24260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:48.303599 24260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:48.303771 24260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:48.308282 24260 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:36187
I20260809 23:09:48.308413 24366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:36187 every 8 connection(s)
I20260809 23:09:48.308746 24260 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:48.309890 24368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:48.312824 24368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c: Bootstrap starting.
I20260809 23:09:48.313578 24368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:48.313882 24368 log.cc:824] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:48.314890 24368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c: No bootstrap required, opened a new log
I20260809 23:09:48.316298 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 24260
I20260809 23:09:48.316423 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:48.316658 8759 external_mini_cluster.cc:1428] Setting key ef2ea31d2bd81feaebf954bee40502ef
I20260809 23:09:48.316920 24368 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1909b8b43d249428e2bf988b735fa5c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36187 } }
I20260809 23:09:48.317119 24368 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:48.317162 24368 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1909b8b43d249428e2bf988b735fa5c, State: Initialized, Role: FOLLOWER
I20260809 23:09:48.317363 24368 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1909b8b43d249428e2bf988b735fa5c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36187 } }
I20260809 23:09:48.317591 24368 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:48.317663 24368 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:48.317761 24368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:48.318670 24368 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1909b8b43d249428e2bf988b735fa5c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36187 } }
I20260809 23:09:48.318809 24368 leader_election.cc:302] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1909b8b43d249428e2bf988b735fa5c; no voters:
I20260809 23:09:48.319067 24368 leader_election.cc:288] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:48.319147 24379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:48.319335 24379 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [term 1 LEADER]: Becoming Leader. State: Replica: a1909b8b43d249428e2bf988b735fa5c, State: Running, Role: LEADER
I20260809 23:09:48.319459 24379 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1909b8b43d249428e2bf988b735fa5c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36187 } }
I20260809 23:09:48.319344 24368 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:48.320665 24383 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1909b8b43d249428e2bf988b735fa5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1909b8b43d249428e2bf988b735fa5c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36187 } } }
I20260809 23:09:48.320760 24383 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:48.320976 24381 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1909b8b43d249428e2bf988b735fa5c. Latest consensus state: current_term: 1 leader_uuid: "a1909b8b43d249428e2bf988b735fa5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1909b8b43d249428e2bf988b735fa5c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36187 } } }
W20260809 23:09:48.321462 24402 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 23:09:48.321796 24381 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c [sys.catalog]: This master's current role is: LEADER
W20260809 23:09:48.321811 24402 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:09:48.321919 24403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:48.322099 24403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:48.324043 24403 catalog_manager.cc:1347] Generated new cluster ID: 66be6f6b8e5540eea0488524cc752ff9
I20260809 23:09:48.324124 24403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:48.324216 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.317632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.8.141.254:36187)
0809 23:09:48.317925 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:48.317935 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:48.317970 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:09:48.318102 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:48.318109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:48.318180 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:48.318341 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:48.318538 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.320493 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.321709 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.322330 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.322464 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.323256 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:48.323260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:48.323260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:48.323484 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:48.323529 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:48.323544 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:48.323620 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:48.323835 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 23:09:48.323925 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":132,"threads_started":1}
I20260809 23:09:48.331966 24403 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:48.332716 24403 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:48.342453 24403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1909b8b43d249428e2bf988b735fa5c: Generated new TSK 0
I20260809 23:09:48.342698 24403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:09:48.344475 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36187
--builtin_ntp_servers=127.8.141.212:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 23:09:48.439015 24408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:48.439281 24408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:48.439325 24408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:48.440591 24408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:48.440662 24408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:48.440709 24408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:48.442250 24408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:48.442549 24408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:48.442893 24408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:48.448542 24414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.448597 24416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.448647 24420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:48.449216 24408 server_base.cc:1034] running on GCE node
I20260809 23:09:48.449515 24408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:48.449796 24408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:48.451615 24408 hybrid_clock.cc:648] HybridClock initialized: now 1786316988451575 us; error 43 us; skew 500 ppm
I20260809 23:09:48.454005 24408 webserver.cc:460] Webserver started at http://127.8.141.193:39605/ using document root <none> and password file <none>
I20260809 23:09:48.454327 24408 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:48.454411 24408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:48.454578 24408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:48.455950 24408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "d6ed4c41975d44d5a6d18a7cb03d6205"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "d8d39b3686679ff4a47b991af7d49aed"
server_key_iv: "3228b772150d0d8ff7f33e80e912d009"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.456562 24408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "d6ed4c41975d44d5a6d18a7cb03d6205"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "d8d39b3686679ff4a47b991af7d49aed"
server_key_iv: "3228b772150d0d8ff7f33e80e912d009"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.458297 24408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 23:09:48.459364 24428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.459592 24408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:48.459690 24408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "d6ed4c41975d44d5a6d18a7cb03d6205"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "d8d39b3686679ff4a47b991af7d49aed"
server_key_iv: "3228b772150d0d8ff7f33e80e912d009"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.459779 24408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:48.481421 24408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:48.481993 24408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:48.482195 24408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:48.482484 24408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:48.482880 24408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:48.482940 24408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.482985 24408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:48.483014 24408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.489765 24408 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:40251
I20260809 23:09:48.489818 24610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:40251 every 8 connection(s)
I20260809 23:09:48.490291 24408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:48.491361 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 24408
I20260809 23:09:48.491483 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:48.491689 8759 external_mini_cluster.cc:1428] Setting key f2f9b11cac4db5de8e51b330ddfeb0c7
I20260809 23:09:48.492749 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36187
--builtin_ntp_servers=127.8.141.212:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 23:09:48.496878 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.491913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:60287 (local address 127.8.141.254:36187)
0809 23:09:48.491976 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:48.491978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:48.491990 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:09:48.492374 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:48.492375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:48.492389 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:48.492459 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:48.493424 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.493932 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.495187 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.495403 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.495492 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.496454 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:48.496456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:48.496457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:48.496463 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:48.496483 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:48.496488 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:48.496558 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:48.496708 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 23:09:48.496752 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:09:48.497321 24615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.491144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36187 (local address 127.8.141.193:60287)
0809 23:09:48.491386 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:48.491435 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:48.492267 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:48.492533 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:48.492537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:48.492548 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:48.493292 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.493301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.494031 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.494032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.495082 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.495085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.495511 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.495513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.495672 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.496289 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:48.496289 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:48.496322 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:48.496347 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:48.496604 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:48.496612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:48.496614 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:09:48.496692 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:09:48.497071 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":133,"threads_started":1}
I20260809 23:09:48.498239 24611 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36187
I20260809 23:09:48.498407 24611 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:48.498641 24611 heartbeater.cc:511] Master 127.8.141.254:36187 requested a full tablet report, sending...
I20260809 23:09:48.499382 24314 ts_manager.cc:194] Registered new tserver with Master: d6ed4c41975d44d5a6d18a7cb03d6205 (127.8.141.193:40251)
I20260809 23:09:48.500051 24314 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.193:60287
W20260809 23:09:48.585812 24616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:48.586084 24616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:48.586125 24616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:48.587857 24616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:48.587962 24616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:48.588013 24616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:48.590430 24616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:48.590924 24616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:48.591329 24616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:48.598115 24634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.598145 24638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.598361 24632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:48.599066 24616 server_base.cc:1034] running on GCE node
I20260809 23:09:48.599428 24616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:48.599855 24616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:48.601080 24616 hybrid_clock.cc:648] HybridClock initialized: now 1786316988601069 us; error 36 us; skew 500 ppm
I20260809 23:09:48.602623 24616 webserver.cc:460] Webserver started at http://127.8.141.194:36665/ using document root <none> and password file <none>
I20260809 23:09:48.602851 24616 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:48.602906 24616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:48.603008 24616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:48.604132 24616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "41d3691a1595453bb0953ca8c388ab00"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "cb62c3feebaf5f4cd5faad92b09e2dd3"
server_key_iv: "f84a18c28183394169d3bc7f07ebf831"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.604553 24616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "41d3691a1595453bb0953ca8c388ab00"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "cb62c3feebaf5f4cd5faad92b09e2dd3"
server_key_iv: "f84a18c28183394169d3bc7f07ebf831"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.606132 24616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:48.607118 24646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.607640 24616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:48.607775 24616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "41d3691a1595453bb0953ca8c388ab00"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "cb62c3feebaf5f4cd5faad92b09e2dd3"
server_key_iv: "f84a18c28183394169d3bc7f07ebf831"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.607898 24616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:48.624037 24616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:48.624429 24616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:48.624585 24616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:48.624864 24616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:48.625255 24616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:48.625312 24616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.625350 24616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:48.625449 24616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.634047 24616 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:39721
I20260809 23:09:48.634121 24827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:39721 every 8 connection(s)
I20260809 23:09:48.634523 24616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:48.639737 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 24616
I20260809 23:09:48.639920 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:48.640147 8759 external_mini_cluster.cc:1428] Setting key e148e9d4c1857566ffd087b89ab407f9
I20260809 23:09:48.641721 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36187
--builtin_ntp_servers=127.8.141.212:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 23:09:48.641811 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.635893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:44311 (local address 127.8.141.254:36187)
0809 23:09:48.636028 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:48.636031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:48.636305 (+ 274us) server_negotiation.cc:407] Connection header received
0809 23:09:48.636827 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:48.636829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:48.636843 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:48.636923 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:48.637985 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.638614 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.639845 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.640207 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.640298 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.641398 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:48.641400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:48.641400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:48.641407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:48.641435 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:48.641440 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:48.641517 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:48.641668 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:09:48.641709 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 23:09:48.642342 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.635811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36187 (local address 127.8.141.194:44311)
0809 23:09:48.636138 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:48.636184 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:48.636702 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:48.637119 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:48.637123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:48.637133 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:48.637719 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.637724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.638806 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.638807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.639704 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.639707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.640317 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.640318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.640517 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.641178 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:48.641178 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:48.641224 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:48.641263 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:48.641548 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:48.641565 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:48.641569 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:48.641648 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 23:09:48.641749 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":143,"threads_started":1}
I20260809 23:09:48.643205 24828 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36187
I20260809 23:09:48.643393 24828 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:48.643677 24828 heartbeater.cc:511] Master 127.8.141.254:36187 requested a full tablet report, sending...
I20260809 23:09:48.644153 24314 ts_manager.cc:194] Registered new tserver with Master: 41d3691a1595453bb0953ca8c388ab00 (127.8.141.194:39721)
I20260809 23:09:48.644744 24314 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.194:44311
W20260809 23:09:48.734495 24833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:48.734763 24833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:48.734789 24833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:48.736088 24833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:48.736141 24833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:48.736171 24833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:48.737900 24833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:48.738186 24833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:48.738525 24833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:48.744246 24843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.744371 24849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.744371 24842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:48.744871 24833 server_base.cc:1034] running on GCE node
I20260809 23:09:48.745103 24833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:48.745464 24833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:48.746657 24833 hybrid_clock.cc:648] HybridClock initialized: now 1786316988746627 us; error 64 us; skew 500 ppm
I20260809 23:09:48.748101 24833 webserver.cc:460] Webserver started at http://127.8.141.195:35093/ using document root <none> and password file <none>
I20260809 23:09:48.748440 24833 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:48.748520 24833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:48.748667 24833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:48.749881 24833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "2a0a71ef14dc4539a2b464baccf72c5a"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "ed73be9d950c13fecdb7a481b9cffac0"
server_key_iv: "7b43c85ab89b0c8744079c768ceebbd3"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.750305 24833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "2a0a71ef14dc4539a2b464baccf72c5a"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "ed73be9d950c13fecdb7a481b9cffac0"
server_key_iv: "7b43c85ab89b0c8744079c768ceebbd3"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.751797 24833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:09:48.752826 24856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.753422 24833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:48.753547 24833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "2a0a71ef14dc4539a2b464baccf72c5a"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "ed73be9d950c13fecdb7a481b9cffac0"
server_key_iv: "7b43c85ab89b0c8744079c768ceebbd3"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.753636 24833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:48.761276 24833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:48.761677 24833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:48.761883 24833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:48.762146 24833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:48.762533 24833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:48.762599 24833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.762646 24833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:48.762676 24833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.769845 24833 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:34901
I20260809 23:09:48.769938 25031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:34901 every 8 connection(s)
I20260809 23:09:48.770318 24833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:48.776199 24377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.771214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:46501 (local address 127.8.141.254:36187)
0809 23:09:48.771293 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:48.771295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:48.771558 (+ 263us) server_negotiation.cc:407] Connection header received
0809 23:09:48.772163 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:48.772164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:48.772179 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:48.772246 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:48.772949 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.773583 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.774621 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.774846 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.774928 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.775872 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:48.775874 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:48.775875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:48.775881 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:48.775898 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:48.775903 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:48.775956 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:48.776072 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 23:09:48.776108 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:09:48.776463 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:48.771181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36187 (local address 127.8.141.195:46501)
0809 23:09:48.771421 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:48.771462 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:48.772040 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:48.772322 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:48.772326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:48.772334 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:48.772851 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.772855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.773689 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.773690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.774516 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:48.774518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:48.774940 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:48.774941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:48.775085 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:48.775709 (+ 624us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:48.775709 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:48.775746 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:48.775771 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:48.775969 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:48.775975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:48.775978 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:48.776066 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:09:48.776179 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":105,"threads_started":1}
I20260809 23:09:48.777161 25032 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36187
I20260809 23:09:48.777273 25032 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:48.777523 25032 heartbeater.cc:511] Master 127.8.141.254:36187 requested a full tablet report, sending...
I20260809 23:09:48.777940 24314 ts_manager.cc:194] Registered new tserver with Master: 2a0a71ef14dc4539a2b464baccf72c5a (127.8.141.195:34901)
I20260809 23:09:48.778287 24314 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.195:46501
I20260809 23:09:48.778860 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 24833
I20260809 23:09:48.779006 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:48.779212 8759 external_mini_cluster.cc:1428] Setting key c75994b7bf2639d4e79d8eab93e5d0ea
I20260809 23:09:48.780308 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:09:48.782413 24377 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:48.781812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.8.141.254:36187)
0809 23:09:48.781878 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:48.781880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:48.782092 (+ 212us) server_negotiation.cc:407] Connection header received
0809 23:09:48.782120 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:48.782121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:48.782133 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:48.782182 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:48.782348 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55676: BlockingRecv error: recv got EOF from 127.0.0.1:55676 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:09:48.782539 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 24408
I20260809 23:09:48.804363 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 24616
I20260809 23:09:48.815326 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 24833
I20260809 23:09:48.836162 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 24260
2026-08-09T23:09:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (705 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T23:09:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:48Z Disabled control of system clock
I20260809 23:09:48.880491 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:45879
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:45879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:09:48.972105 25047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:48.972362 25047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:48.972388 25047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:48.974123 25047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:48.974198 25047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:48.974212 25047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:48.974226 25047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:48.974241 25047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:48.976080 25047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:45879
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:45879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:48.976344 25047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:48.976634 25047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:48.982394 25061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.982405 25059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:48.982405 25063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:48.982694 25047 server_base.cc:1034] running on GCE node
I20260809 23:09:48.983014 25047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:48.983357 25047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:48.984577 25047 hybrid_clock.cc:648] HybridClock initialized: now 1786316988984534 us; error 71 us; skew 500 ppm
I20260809 23:09:48.986143 25047 webserver.cc:460] Webserver started at http://127.8.141.254:37401/ using document root <none> and password file <none>
I20260809 23:09:48.986472 25047 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:48.986567 25047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:48.986732 25047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:48.987918 25047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "242fa4067b0d4330bd7b5397545d7ff7"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "09abd4ab3b9d911a4122655c1cff688d"
server_key_iv: "996afa9894c8e958a93e3440206adafb"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.988319 25047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "242fa4067b0d4330bd7b5397545d7ff7"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "09abd4ab3b9d911a4122655c1cff688d"
server_key_iv: "996afa9894c8e958a93e3440206adafb"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.989946 25047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:09:48.990875 25071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:48.991194 25047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:48.991302 25047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "242fa4067b0d4330bd7b5397545d7ff7"
format_stamp: "Formatted at 2026-08-09 23:09:48 on dist-test-slave-j2vl"
server_key: "09abd4ab3b9d911a4122655c1cff688d"
server_key_iv: "996afa9894c8e958a93e3440206adafb"
server_key_version: "encryptionkey@0"
I20260809 23:09:48.991396 25047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:48.995129 25047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:48.995432 25047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:48.995594 25047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:49.000336 25047 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:45879
I20260809 23:09:49.000371 25161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:45879 every 8 connection(s)
I20260809 23:09:49.000751 25047 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:49.001708 25162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:49.004792 25162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7: Bootstrap starting.
I20260809 23:09:49.005524 25162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:49.005878 25162 log.cc:824] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:49.005935 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 25047
I20260809 23:09:49.006089 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:49.006333 8759 external_mini_cluster.cc:1428] Setting key 2381fe8111b7bb306b084f7636d542a7
I20260809 23:09:49.006892 25162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7: No bootstrap required, opened a new log
I20260809 23:09:49.008970 25162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fa4067b0d4330bd7b5397545d7ff7" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 45879 } }
I20260809 23:09:49.009138 25162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:49.009163 25162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 242fa4067b0d4330bd7b5397545d7ff7, State: Initialized, Role: FOLLOWER
I20260809 23:09:49.009353 25162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fa4067b0d4330bd7b5397545d7ff7" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 45879 } }
I20260809 23:09:49.009481 25162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:49.009524 25162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:49.009615 25162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:49.010687 25162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fa4067b0d4330bd7b5397545d7ff7" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 45879 } }
I20260809 23:09:49.010885 25162 leader_election.cc:302] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 242fa4067b0d4330bd7b5397545d7ff7; no voters:
I20260809 23:09:49.011080 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:49.007081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60264 (local address 127.8.141.254:45879)
0809 23:09:49.007293 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:49.007301 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:49.007330 (+ 29us) server_negotiation.cc:407] Connection header received
0809 23:09:49.008989 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:49.008999 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:49.009051 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:49.009261 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:49.010354 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:49.010358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:49.010358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:49.010364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:49.010410 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:49.010440 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:49.010521 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:49.010723 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:09:49.010800 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260809 23:09:49.011122 25162 leader_election.cc:288] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:49.011422 25162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:49.011462 25171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:49.011668 25171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [term 1 LEADER]: Becoming Leader. State: Replica: 242fa4067b0d4330bd7b5397545d7ff7, State: Running, Role: LEADER
I20260809 23:09:49.011821 25171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fa4067b0d4330bd7b5397545d7ff7" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 45879 } }
I20260809 23:09:49.012296 25172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "242fa4067b0d4330bd7b5397545d7ff7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fa4067b0d4330bd7b5397545d7ff7" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 45879 } } }
I20260809 23:09:49.012303 25173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 242fa4067b0d4330bd7b5397545d7ff7. Latest consensus state: current_term: 1 leader_uuid: "242fa4067b0d4330bd7b5397545d7ff7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "242fa4067b0d4330bd7b5397545d7ff7" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 45879 } } }
I20260809 23:09:49.012450 25173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:49.012400 25172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7 [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:49.012676 25186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:49.013280 25186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:49.015538 25186 catalog_manager.cc:1347] Generated new cluster ID: f95c3d7288cf437e891a4ed8f99c1053
I20260809 23:09:49.015611 25186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:09:49.035928 25186 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:49.036177 25186 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:49.039090 25186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 242fa4067b0d4330bd7b5397545d7ff7: Generated new TSK 0
I20260809 23:09:49.039338 25186 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:09:49.042851 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:45879
--builtin_ntp_servers=127.8.141.212:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:09:49.139405 25199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:49.139642 25199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:49.139670 25199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:49.141300 25199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:49.141440 25199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:49.141485 25199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:49.143543 25199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:45879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:49.143873 25199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:49.144237 25199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:49.150280 25209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:49.150477 25210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:49.150548 25218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:49.150825 25199 server_base.cc:1034] running on GCE node
I20260809 23:09:49.151155 25199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:49.151475 25199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:49.152720 25199 hybrid_clock.cc:648] HybridClock initialized: now 1786316989152704 us; error 34 us; skew 500 ppm
I20260809 23:09:49.154810 25199 webserver.cc:460] Webserver started at http://127.8.141.193:42111/ using document root <none> and password file <none>
I20260809 23:09:49.155108 25199 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:49.155187 25199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:49.155349 25199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:49.156515 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "1ad69cef8fca4ac3a81a3c97738cf8e6"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "8c84cf9d0870f8d26756b281eba016a2"
server_key_iv: "2bb0c2b41f4344bdae002fe4db089139"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.156926 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "1ad69cef8fca4ac3a81a3c97738cf8e6"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "8c84cf9d0870f8d26756b281eba016a2"
server_key_iv: "2bb0c2b41f4344bdae002fe4db089139"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.159226 25199 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 23:09:49.160117 25229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.160377 25199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:49.160504 25199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "1ad69cef8fca4ac3a81a3c97738cf8e6"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "8c84cf9d0870f8d26756b281eba016a2"
server_key_iv: "2bb0c2b41f4344bdae002fe4db089139"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.160594 25199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:49.164249 25199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:49.164561 25199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:49.164714 25199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:49.164955 25199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:49.165305 25199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:49.165383 25199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.165432 25199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:49.165464 25199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.171886 25199 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:41451
I20260809 23:09:49.172005 25420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:41451 every 8 connection(s)
I20260809 23:09:49.172339 25199 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:09:49.176438 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:49.173246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:48167 (local address 127.8.141.254:45879)
0809 23:09:49.173318 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:49.173321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:49.173741 (+ 420us) server_negotiation.cc:407] Connection header received
0809 23:09:49.174271 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:49.174272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:49.174281 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:49.174348 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:49.176090 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:49.176091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:49.176091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:49.176096 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:49.176119 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:49.176124 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:49.176183 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:49.176348 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 23:09:49.176376 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:09:49.176631 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:49.173236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:45879 (local address 127.8.141.193:48167)
0809 23:09:49.173563 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:49.173612 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:49.174130 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:49.174610 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:49.174614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:49.174624 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:49.175873 (+ 1249us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:49.175874 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:49.175909 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:49.175945 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:49.176215 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:49.176223 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:49.176227 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:49.176313 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 23:09:49.176373 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":154,"threads_started":1}
I20260809 23:09:49.177673 25422 heartbeater.cc:347] Connected to a master server at 127.8.141.254:45879
I20260809 23:09:49.177826 25422 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:49.178043 25422 heartbeater.cc:511] Master 127.8.141.254:45879 requested a full tablet report, sending...
I20260809 23:09:49.179087 25099 ts_manager.cc:194] Registered new tserver with Master: 1ad69cef8fca4ac3a81a3c97738cf8e6 (127.8.141.193:41451)
I20260809 23:09:49.179498 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 25199
I20260809 23:09:49.179616 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:49.179853 8759 external_mini_cluster.cc:1428] Setting key a6aee5b7225ad2f84d7c98abc18a3c88
I20260809 23:09:49.180919 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:45879
--builtin_ntp_servers=127.8.141.212:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:09:49.270854 25425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:49.271173 25425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:49.271217 25425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:49.273033 25425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:49.273141 25425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:49.273196 25425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:49.275130 25425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:45879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:49.275471 25425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:49.275864 25425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:09:49.281834 25425 server_base.cc:1034] running on GCE node
W20260809 23:09:49.281849 25436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:49.281860 25434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:49.282081 25439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:49.282411 25425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:49.282778 25425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:49.283949 25425 hybrid_clock.cc:648] HybridClock initialized: now 1786316989283914 us; error 51 us; skew 500 ppm
I20260809 23:09:49.285665 25425 webserver.cc:460] Webserver started at http://127.8.141.194:41305/ using document root <none> and password file <none>
I20260809 23:09:49.285924 25425 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:49.285974 25425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:49.286070 25425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:49.287182 25425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "c87501c2f9a84f038c13a6ba76f345d6"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "0a6509a65c5365c3b5c0f8279a1e7aba"
server_key_iv: "0b1a66a5e6e755fa06aa42de7d75f2a6"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.287513 25425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "c87501c2f9a84f038c13a6ba76f345d6"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "0a6509a65c5365c3b5c0f8279a1e7aba"
server_key_iv: "0b1a66a5e6e755fa06aa42de7d75f2a6"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.288873 25425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:49.289747 25447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.289983 25425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:09:49.290081 25425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "c87501c2f9a84f038c13a6ba76f345d6"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "0a6509a65c5365c3b5c0f8279a1e7aba"
server_key_iv: "0b1a66a5e6e755fa06aa42de7d75f2a6"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.290144 25425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:49.293694 25425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:49.294026 25425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:49.294135 25425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:49.294305 25425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:49.294632 25425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:49.294670 25425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.294691 25425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:49.294704 25425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.300626 25425 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:40839
I20260809 23:09:49.300845 25644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:40839 every 8 connection(s)
I20260809 23:09:49.301113 25425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:49.304287 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:49.301981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:46197 (local address 127.8.141.254:45879)
0809 23:09:49.302034 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:49.302036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:49.302246 (+ 210us) server_negotiation.cc:407] Connection header received
0809 23:09:49.302709 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:49.302710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:49.302715 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:49.302787 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:49.304054 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:49.304055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:49.304056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:49.304061 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:49.304077 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:49.304082 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:49.304154 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:49.304205 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 23:09:49.304224 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:49.304497 25648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:49.301920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:45879 (local address 127.8.141.194:46197)
0809 23:09:49.302109 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:49.302152 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:49.302623 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:49.302842 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:49.302845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:49.302854 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:49.303882 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:49.303882 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:49.303912 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:49.303941 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:49.304152 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:49.304155 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:49.304157 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:09:49.304206 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 23:09:49.304257 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260809 23:09:49.305068 25645 heartbeater.cc:347] Connected to a master server at 127.8.141.254:45879
I20260809 23:09:49.305176 25645 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:49.305316 25645 heartbeater.cc:511] Master 127.8.141.254:45879 requested a full tablet report, sending...
I20260809 23:09:49.305800 25099 ts_manager.cc:194] Registered new tserver with Master: c87501c2f9a84f038c13a6ba76f345d6 (127.8.141.194:40839)
I20260809 23:09:49.307708 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 25425
I20260809 23:09:49.307857 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:09:49.308071 8759 external_mini_cluster.cc:1428] Setting key 204f238c76794fe99fead20db0345090
I20260809 23:09:49.309069 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:45879
--builtin_ntp_servers=127.8.141.212:38831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 23:09:49.399638 25649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:49.399887 25649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:49.399914 25649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:49.401180 25649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:49.401230 25649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:49.401257 25649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:49.402789 25649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:45879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:49.403064 25649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:49.403568 25649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:49.409160 25658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:49.409183 25659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:49.409183 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:49.409929 25649 server_base.cc:1034] running on GCE node
I20260809 23:09:49.410164 25649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:49.410414 25649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:49.411624 25649 hybrid_clock.cc:648] HybridClock initialized: now 1786316989411599 us; error 36 us; skew 500 ppm
I20260809 23:09:49.413267 25649 webserver.cc:460] Webserver started at http://127.8.141.195:46821/ using document root <none> and password file <none>
I20260809 23:09:49.413606 25649 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:49.413659 25649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:49.413816 25649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:49.415016 25649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "707d5cb60904431591a7f9ffb04d1fb3"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "c4c93fb13e2b19da8c068b15d9877021"
server_key_iv: "88d322b061deb8886b885a9016c6de08"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.415352 25649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "707d5cb60904431591a7f9ffb04d1fb3"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "c4c93fb13e2b19da8c068b15d9877021"
server_key_iv: "88d322b061deb8886b885a9016c6de08"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.416826 25649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:49.417856 25675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.418195 25649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:09:49.418315 25649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "707d5cb60904431591a7f9ffb04d1fb3"
format_stamp: "Formatted at 2026-08-09 23:09:49 on dist-test-slave-j2vl"
server_key: "c4c93fb13e2b19da8c068b15d9877021"
server_key_iv: "88d322b061deb8886b885a9016c6de08"
server_key_version: "encryptionkey@0"
I20260809 23:09:49.418404 25649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:49.422377 25649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:49.422688 25649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:49.422863 25649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:49.423107 25649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:49.423458 25649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:49.423516 25649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.423561 25649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:49.423589 25649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:49.430159 25649 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:42487
I20260809 23:09:49.430214 25848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:42487 every 8 connection(s)
I20260809 23:09:49.430568 25649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:49.434257 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:49.431431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:42503 (local address 127.8.141.254:45879)
0809 23:09:49.431507 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:49.431509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:49.431717 (+ 208us) server_negotiation.cc:407] Connection header received
0809 23:09:49.432254 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:49.432255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:49.432262 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:49.432318 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:49.433927 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:49.433929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:49.433929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:09:49.433935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:49.433960 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:09:49.433965 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:49.434042 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:49.434151 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 23:09:49.434171 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:09:49.434520 25856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:49.431346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:45879 (local address 127.8.141.195:42503)
0809 23:09:49.431584 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:49.431630 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:49.432135 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:49.432417 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:49.432422 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:49.432431 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:49.433739 (+ 1308us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:09:49.433739 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:49.433773 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:09:49.433808 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:49.434039 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:49.434048 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:49.434051 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:09:49.434163 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 23:09:49.434245 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260809 23:09:49.435061 25850 heartbeater.cc:347] Connected to a master server at 127.8.141.254:45879
I20260809 23:09:49.435151 25850 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:49.435432 25850 heartbeater.cc:511] Master 127.8.141.254:45879 requested a full tablet report, sending...
I20260809 23:09:49.435467 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 25649
I20260809 23:09:49.435586 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:49.435863 8759 external_mini_cluster.cc:1428] Setting key eee3159b140133f0a62ca13ff3ad5a0b
I20260809 23:09:49.435889 25099 ts_manager.cc:194] Registered new tserver with Master: 707d5cb60904431591a7f9ffb04d1fb3 (127.8.141.195:42487)
I20260809 23:09:49.436806 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 23:09:49.439157 25167 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:09:49.438592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60268 (local address 127.8.141.254:45879)
0809 23:09:49.438787 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:49.438789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:49.438807 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:09:49.438873 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:49.438873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:49.438878 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:49.438936 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:49.439086 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60268: BlockingRecv error: recv got EOF from 127.0.0.1:60268 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
I20260809 23:09:49.439214 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 25199
I20260809 23:09:49.472990 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 25425
I20260809 23:09:49.488796 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 25649
I20260809 23:09:49.508466 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 25047
2026-08-09T23:09:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (667 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:49 dist-test-slave-j2vl krb5kdc[25868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:49 dist-test-slave-j2vl krb5kdc[25868](info): set up 2 sockets
Aug 09 23:09:49 dist-test-slave-j2vl krb5kdc[25868](info): commencing operation
krb5kdc: starting...
W20260809 23:09:50.829725 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:50 dist-test-slave-j2vl krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:09:50.966158 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36607
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:41481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:36607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:51.053887 25890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:51.054109 25890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:51.054136 25890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:51.055372 25890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:09:51.055420 25890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:51.055435 25890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:51.055449 25890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:09:51.055505 25890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:09:51.057085 25890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:36607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:51.057356 25890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:51.057642 25890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:51.062997 25898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.062997 25895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.063194 25904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:51.063210 25890 server_base.cc:1034] running on GCE node
I20260809 23:09:51.063539 25890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:51.063894 25890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:51.065056 25890 hybrid_clock.cc:648] HybridClock initialized: now 1786316991065021 us; error 48 us; skew 500 ppm
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316991, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:51.068059 25890 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:09:51.068585 25890 webserver.cc:460] Webserver started at http://127.8.141.254:36909/ using document root <none> and password file <none>
I20260809 23:09:51.068862 25890 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:51.069024 25890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:51.069255 25890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:51.070439 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "02f71332736f43c48fee489ca654527f"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "df997f225d913e823d901e8627a37ade"
server_key_iv: "4e48a1027188da04814555ddac2e6800"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.070833 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "02f71332736f43c48fee489ca654527f"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "df997f225d913e823d901e8627a37ade"
server_key_iv: "4e48a1027188da04814555ddac2e6800"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.072293 25890 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:51.073181 25912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.073493 25890 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:51.073602 25890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "02f71332736f43c48fee489ca654527f"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "df997f225d913e823d901e8627a37ade"
server_key_iv: "4e48a1027188da04814555ddac2e6800"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.073693 25890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:51.085310 25890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:51.087656 25890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:51.087860 25890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:51.092353 25890 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:36607
I20260809 23:09:51.092439 25993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:36607 every 8 connection(s)
I20260809 23:09:51.092823 25890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:09:51.094063 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:51.096510 25999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f: Bootstrap starting.
I20260809 23:09:51.097106 25999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:51.097460 25999 log.cc:824] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:51.098272 25999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f: No bootstrap required, opened a new log
I20260809 23:09:51.099999 25999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f71332736f43c48fee489ca654527f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36607 } }
I20260809 23:09:51.100152 25999 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:51.100175 25999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f71332736f43c48fee489ca654527f, State: Initialized, Role: FOLLOWER
I20260809 23:09:51.100349 25999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f71332736f43c48fee489ca654527f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36607 } }
I20260809 23:09:51.100446 25999 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:09:51.100471 25999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:09:51.100502 25999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:51.101475 25999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f71332736f43c48fee489ca654527f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36607 } }
I20260809 23:09:51.101572 25999 leader_election.cc:302] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02f71332736f43c48fee489ca654527f; no voters:
I20260809 23:09:51.101717 25999 leader_election.cc:288] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:09:51.101799 26005 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:51.101974 26005 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [term 1 LEADER]: Becoming Leader. State: Replica: 02f71332736f43c48fee489ca654527f, State: Running, Role: LEADER
I20260809 23:09:51.102017 25999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:09:51.102075 26005 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f71332736f43c48fee489ca654527f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36607 } }
I20260809 23:09:51.102509 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 25890
I20260809 23:09:51.102475 26008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02f71332736f43c48fee489ca654527f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f71332736f43c48fee489ca654527f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36607 } } }
I20260809 23:09:51.102581 26008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:51.102645 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:09:51.102774 26009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02f71332736f43c48fee489ca654527f. Latest consensus state: current_term: 1 leader_uuid: "02f71332736f43c48fee489ca654527f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f71332736f43c48fee489ca654527f" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36607 } } }
I20260809 23:09:51.102793 26016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:09:51.102859 26009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f [sys.catalog]: This master's current role is: LEADER
I20260809 23:09:51.102913 8759 external_mini_cluster.cc:1428] Setting key f5b3550877bb14a817ba34ac0d8950f4
I20260809 23:09:51.103520 26016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:09:51.105633 26016 catalog_manager.cc:1347] Generated new cluster ID: 24de22e3d7f34a668fa847b5e38f0d9a
I20260809 23:09:51.105690 26016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:51.116470 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.103587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.8.141.254:36607)
0809 23:09:51.103779 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.103789 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.104081 (+ 292us) server_negotiation.cc:407] Connection header received
0809 23:09:51.104207 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.104214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.104272 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.104409 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:51.105227 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.106634 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.107842 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.108579 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.108725 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.112065 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:51.112070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:51.112070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:51.112087 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:51.114370 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.114900 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.114904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.114904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.115038 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.115333 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.115335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.115335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.115729 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:51.115923 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.116002 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.116188 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260809 23:09:51.126295 26016 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:09:51.127107 26016 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:09:51.142911 26016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f: Generated new TSK 0
I20260809 23:09:51.143172 26016 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:09:51.181146 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36607
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:41481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:51.272015 26048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:51.272255 26048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:51.272284 26048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:51.273773 26048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:51.273835 26048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:51.273866 26048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:09:51.275483 26048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:51.275702 26048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:51.275945 26048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:51.281596 26060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.281844 26058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.281684 26056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:51.282100 26048 server_base.cc:1034] running on GCE node
I20260809 23:09:51.282316 26048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:51.282578 26048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:51.283771 26048 hybrid_clock.cc:648] HybridClock initialized: now 1786316991283744 us; error 36 us; skew 500 ppm
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316991, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:51.286722 26048 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:09:51.287338 26048 webserver.cc:460] Webserver started at http://127.8.141.193:45305/ using document root <none> and password file <none>
I20260809 23:09:51.287544 26048 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:51.287586 26048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:51.287684 26048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:51.288751 26048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "cd73327b96c7474a9779c84aaa52be57"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "feb339aed7b9c93e5a066cb2ef056044"
server_key_iv: "de5ba91e2e6211178f3cfce0473a64b7"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.289062 26048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "cd73327b96c7474a9779c84aaa52be57"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "feb339aed7b9c93e5a066cb2ef056044"
server_key_iv: "de5ba91e2e6211178f3cfce0473a64b7"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.290792 26048 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 23:09:51.291682 26070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.292024 26048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:09:51.292116 26048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "cd73327b96c7474a9779c84aaa52be57"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "feb339aed7b9c93e5a066cb2ef056044"
server_key_iv: "de5ba91e2e6211178f3cfce0473a64b7"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.292232 26048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:51.307529 26048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:51.310075 26048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:51.310256 26048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:51.310554 26048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:51.310918 26048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:51.310956 26048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.310999 26048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:51.311026 26048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.317727 26048 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:42253
I20260809 23:09:51.317806 26264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:42253 every 8 connection(s)
I20260809 23:09:51.318151 26048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316991, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:51.327957 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 26048
I20260809 23:09:51.328075 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:09:51.328298 8759 external_mini_cluster.cc:1428] Setting key d4991384fd93e314702c4698c52f4a6e
I20260809 23:09:51.328665 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.319058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:57327 (local address 127.8.141.254:36607)
0809 23:09:51.319122 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.319125 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.319303 (+ 178us) server_negotiation.cc:407] Connection header received
0809 23:09:51.319847 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.319848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.319864 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.319932 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:51.320987 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.321589 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.322593 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.322847 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.322917 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.325436 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:51.325438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:51.325438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:51.325446 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:51.326986 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.327432 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.327433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.327433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.327499 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.327808 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.327810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.327810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.328000 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:51.328100 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.328453 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.328564 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 23:09:51.328876 26270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.318984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36607 (local address 127.8.141.193:57327)
0809 23:09:51.319165 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.319210 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.319705 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.320047 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.320051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.320658 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:51.320881 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.320886 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.321677 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.321678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.322503 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.322505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.322949 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.322950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.323057 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.323732 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:51.323741 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:51.325245 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:51.327073 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:51.327082 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:51.327084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:51.327324 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:51.327586 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:51.327588 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:51.327588 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:51.327716 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:51.328111 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:51.328120 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:51.328264 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.328404 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.328609 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260809 23:09:51.329532 26265 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36607
I20260809 23:09:51.329667 26265 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:51.329886 26265 heartbeater.cc:511] Master 127.8.141.254:36607 requested a full tablet report, sending...
I20260809 23:09:51.330506 25936 ts_manager.cc:194] Registered new tserver with Master: cd73327b96c7474a9779c84aaa52be57 (127.8.141.193:42253)
I20260809 23:09:51.331138 25936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:57327
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:09:51.360934 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36607
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:41481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:51.454823 26277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:51.455117 26277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:51.455168 26277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:51.456583 26277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:51.456669 26277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:51.456735 26277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:09:51.458772 26277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:51.459210 26277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:51.459759 26277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:51.466898 26290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.467160 26294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.466898 26291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:51.473037 26277 server_base.cc:1034] running on GCE node
I20260809 23:09:51.473454 26277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:51.473811 26277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:51.475137 26277 hybrid_clock.cc:648] HybridClock initialized: now 1786316991475103 us; error 51 us; skew 500 ppm
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316991, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:51.478296 26277 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:09:51.478842 26277 webserver.cc:460] Webserver started at http://127.8.141.194:41303/ using document root <none> and password file <none>
I20260809 23:09:51.479139 26277 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:51.479221 26277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:51.479411 26277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:51.480655 26277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "a33eb9386c84459499f5c15ea06f6ff2"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "ce1ffcbe13ff34874cfd502074ba79e1"
server_key_iv: "24a492ed462280aef8f709e5e4c7ceec"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.481097 26277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "a33eb9386c84459499f5c15ea06f6ff2"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "ce1ffcbe13ff34874cfd502074ba79e1"
server_key_iv: "24a492ed462280aef8f709e5e4c7ceec"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.482831 26277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 23:09:51.483777 26309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.484124 26277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:51.484247 26277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "a33eb9386c84459499f5c15ea06f6ff2"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "ce1ffcbe13ff34874cfd502074ba79e1"
server_key_iv: "24a492ed462280aef8f709e5e4c7ceec"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.484350 26277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:51.500342 26277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:51.503039 26277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:51.503209 26277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:51.503466 26277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:51.503836 26277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:51.503899 26277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.503944 26277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:51.503973 26277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.511052 26277 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:37777
I20260809 23:09:51.511101 26486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:37777 every 8 connection(s)
I20260809 23:09:51.511488 26277 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:09:51.518738 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 26277
I20260809 23:09:51.518888 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316991, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:51.519122 8759 external_mini_cluster.cc:1428] Setting key e435d69439d51ead66d77a0a5e9053cb
I20260809 23:09:51.523758 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.512384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:54065 (local address 127.8.141.254:36607)
0809 23:09:51.512483 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.512485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.512683 (+ 198us) server_negotiation.cc:407] Connection header received
0809 23:09:51.513192 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.513193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.513205 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.513279 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:51.514406 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.514988 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.516309 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.516523 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.516592 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.519598 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:51.519600 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:51.519600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:51.519610 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:51.521888 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.522590 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.522592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.522593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.522687 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.522985 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.522987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.522987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.523156 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:51.523234 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.523527 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.523636 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:09:51.523814 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.512344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36607 (local address 127.8.141.194:54065)
0809 23:09:51.512514 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.512560 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.513087 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.513394 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.513398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.514027 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:51.514246 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.514250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.515098 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.515099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.516202 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.516206 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.516632 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.516633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.516788 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.517479 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:51.517496 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:51.519467 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:51.522000 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:51.522013 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:51.522016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:51.522457 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:51.522771 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:51.522772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:51.522772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:51.522913 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:51.523228 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:51.523233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:51.523387 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.523479 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.523620 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260809 23:09:51.524397 26487 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36607
I20260809 23:09:51.524557 26487 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:51.524787 26487 heartbeater.cc:511] Master 127.8.141.254:36607 requested a full tablet report, sending...
I20260809 23:09:51.525468 25936 ts_manager.cc:194] Registered new tserver with Master: a33eb9386c84459499f5c15ea06f6ff2 (127.8.141.194:37777)
I20260809 23:09:51.526054 25936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:54065
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:09:51.552899 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36607
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:41481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:09:51.640982 26493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:09:51.641273 26493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:09:51.641325 26493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:09:51.642659 26493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:09:51.642738 26493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:09:51.642784 26493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:09:51.644451 26493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:09:51.644770 26493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:09:51.645040 26493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:09:51.650978 26506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.650997 26507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:09:51.650995 26512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:09:51.651295 26493 server_base.cc:1034] running on GCE node
I20260809 23:09:51.651602 26493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:09:51.652153 26493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:09:51.653421 26493 hybrid_clock.cc:648] HybridClock initialized: now 1786316991653372 us; error 67 us; skew 500 ppm
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316991, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:09:51.656443 26493 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:09:51.656989 26493 webserver.cc:460] Webserver started at http://127.8.141.195:34137/ using document root <none> and password file <none>
I20260809 23:09:51.657243 26493 fs_manager.cc:359] Metadata directory not provided
I20260809 23:09:51.657318 26493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:09:51.657536 26493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:09:51.658648 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "b32c25c2d8ba412789415038f0353e1c"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "f448ec0861dc9c015c66beabd7e14999"
server_key_iv: "e861cbb997f77eddd6337b0497036a36"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.659045 26493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "b32c25c2d8ba412789415038f0353e1c"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "f448ec0861dc9c015c66beabd7e14999"
server_key_iv: "e861cbb997f77eddd6337b0497036a36"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.660513 26493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:09:51.661490 26524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.661681 26493 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:09:51.661834 26493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "b32c25c2d8ba412789415038f0353e1c"
format_stamp: "Formatted at 2026-08-09 23:09:51 on dist-test-slave-j2vl"
server_key: "f448ec0861dc9c015c66beabd7e14999"
server_key_iv: "e861cbb997f77eddd6337b0497036a36"
server_key_version: "encryptionkey@0"
I20260809 23:09:51.661932 26493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:09:51.673833 26493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:09:51.676185 26493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:09:51.676337 26493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:09:51.676764 26493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:09:51.677145 26493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:09:51.677184 26493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.677209 26493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:09:51.677222 26493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:09:51.684289 26493 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:34721
I20260809 23:09:51.684343 26701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:34721 every 8 connection(s)
I20260809 23:09:51.684721 26493 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:09:51.689244 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 26493
I20260809 23:09:51.689380 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:09:51.689579 8759 external_mini_cluster.cc:1428] Setting key de62c6224bf6b62b764c9481fdcb63b3
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[25868](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316991, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:09:51.694648 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.685640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:34013 (local address 127.8.141.254:36607)
0809 23:09:51.685692 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.685694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.685920 (+ 226us) server_negotiation.cc:407] Connection header received
0809 23:09:51.686496 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.686497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.686512 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.686579 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:51.687632 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.688186 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.689192 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.689499 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.689579 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.691944 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:51.691946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:51.691946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:51.691954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:51.693235 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.693764 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.693766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.693766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.693826 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.694061 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.694061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.694062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.694173 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:51.694226 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.694507 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.694580 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:51.694900 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.685565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36607 (local address 127.8.141.195:34013)
0809 23:09:51.685781 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.685821 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.686383 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.686654 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.686663 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.687319 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:09:51.687537 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.687542 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.688290 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.688292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.689099 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.689101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.689590 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.689592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.689767 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.690381 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:09:51.690392 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:09:51.691856 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:09:51.693423 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:51.693432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:51.693434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:51.693687 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:51.693890 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:09:51.693891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:09:51.693891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:09:51.694004 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:09:51.694231 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:09:51.694236 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:09:51.694363 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.694506 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.694665 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
I20260809 23:09:51.695434 26702 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36607
I20260809 23:09:51.695549 26702 heartbeater.cc:464] Registering TS with master...
I20260809 23:09:51.695761 26702 heartbeater.cc:511] Master 127.8.141.254:36607 requested a full tablet report, sending...
I20260809 23:09:51.696161 25936 ts_manager.cc:194] Registered new tserver with Master: b32c25c2d8ba412789415038f0353e1c (127.8.141.195:34721)
I20260809 23:09:51.696539 25936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:34013
I20260809 23:09:51.701520 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:09:51.709828 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.703183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.8.141.254:36607)
0809 23:09:51.703268 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.703270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.703465 (+ 195us) server_negotiation.cc:407] Connection header received
0809 23:09:51.703502 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.703503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.703515 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.703581 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:09:51.704283 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.704814 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.705614 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.705832 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.705904 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.707170 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:09:51.707171 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:09:51.707171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:09:51.707179 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:09:51.708639 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.709008 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.709009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.709009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.709067 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:09:51.709244 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:09:51.709245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:09:51.709245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:09:51.709483 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:09:51.709564 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.709665 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.709753 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:09:51.711668 25936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:09:51.713238 25936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:09:51.720819 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.716032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.8.141.193:42253)
0809 23:09:51.716266 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.716272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.716381 (+ 109us) server_negotiation.cc:407] Connection header received
0809 23:09:51.716476 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.716479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.716498 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.716657 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:51.716782 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.717681 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.719407 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.720221 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.720290 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.720352 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.720673 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.720722 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":108,"threads_started":1}
I20260809 23:09:51.720820 26722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.715881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:42253 (local address 127.0.0.1:37194)
0809 23:09:51.716272 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.716299 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.716371 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.716628 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.716632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.716634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:51.716719 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.716722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.717817 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.717819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.719239 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.719241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.720307 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.720308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.720426 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.720528 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.720643 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.720718 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":8,"thread_start_us":71,"threads_started":1}
I20260809 23:09:51.721499 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.715880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:34721 (local address 127.0.0.1:36168)
0809 23:09:51.716250 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.716299 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.716371 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.716768 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.716769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.716771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:51.716820 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.716821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.718034 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.718035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.719233 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.719238 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.721217 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.721218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.721287 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.721291 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.721390 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.721440 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":9,"thread_start_us":137,"threads_started":1}
I20260809 23:09:51.721871 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.716042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.8.141.195:34721)
0809 23:09:51.716588 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.716593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.716603 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:09:51.716646 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.716648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.716663 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.716896 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:51.716980 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.717919 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.720190 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.720865 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.721213 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.721289 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.721724 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.721785 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":92,"threads_started":1}
I20260809 23:09:51.721832 26161 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 (DEFAULT_TABLE table=test-table [id=febcec42cca7434d95ed5e6938eb63d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:51.722278 26161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:51.722606 26596 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 (DEFAULT_TABLE table=test-table [id=febcec42cca7434d95ed5e6938eb63d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:51.722980 26596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:51.723066 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.716144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37777 (local address 127.0.0.1:53966)
0809 23:09:51.717453 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.717466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.717532 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.717712 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.717713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.717715 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:51.717767 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.717769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.720202 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.720203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.721601 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.721605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.722743 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.722744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.722829 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.722835 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.722905 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.722960 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286,"thread_start_us":69,"threads_started":1}
I20260809 23:09:51.723449 26726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.716458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53966 (local address 127.8.141.194:37777)
0809 23:09:51.716872 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.716877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.717547 (+ 670us) server_negotiation.cc:407] Connection header received
0809 23:09:51.717599 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.717601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.717618 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.717709 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:51.718657 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.720016 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.721717 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.722542 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.722629 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.722696 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.723268 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.723336 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":98,"threads_started":1}
I20260809 23:09:51.724277 26385 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 (DEFAULT_TABLE table=test-table [id=febcec42cca7434d95ed5e6938eb63d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:09:51.724648 26385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:09:51.726606 26729 tablet_bootstrap.cc:492] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Bootstrap starting.
I20260809 23:09:51.727272 26729 tablet_bootstrap.cc:654] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:51.727787 26729 log.cc:824] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:51.728592 26729 tablet_bootstrap.cc:492] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: No bootstrap required, opened a new log
I20260809 23:09:51.728684 26729 ts_tablet_manager.cc:1397] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:09:51.728999 26732 tablet_bootstrap.cc:492] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Bootstrap starting.
I20260809 23:09:51.729667 26732 tablet_bootstrap.cc:654] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:51.730228 26732 log.cc:824] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:51.730396 26729 raft_consensus.cc:348] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.730675 26729 raft_consensus.cc:374] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:51.730760 26729 raft_consensus.cc:729] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32c25c2d8ba412789415038f0353e1c, State: Initialized, Role: FOLLOWER
I20260809 23:09:51.731007 26729 consensus_queue.cc:260] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.731300 26732 tablet_bootstrap.cc:492] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: No bootstrap required, opened a new log
I20260809 23:09:51.731379 26729 ts_tablet_manager.cc:1428] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:09:51.731385 26732 ts_tablet_manager.cc:1397] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:09:51.731487 26702 heartbeater.cc:503] Master 127.8.141.254:36607 was elected leader, sending a full tablet report...
I20260809 23:09:51.731671 26731 tablet_bootstrap.cc:492] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57: Bootstrap starting.
I20260809 23:09:51.732475 26731 tablet_bootstrap.cc:654] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57: Neither blocks nor log segments found. Creating new log.
I20260809 23:09:51.733069 26731 log.cc:824] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57: Log is configured to *not* fsync() on all Append() calls
I20260809 23:09:51.733776 26732 raft_consensus.cc:348] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.733968 26731 tablet_bootstrap.cc:492] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57: No bootstrap required, opened a new log
I20260809 23:09:51.733968 26732 raft_consensus.cc:374] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:51.734012 26732 raft_consensus.cc:729] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a33eb9386c84459499f5c15ea06f6ff2, State: Initialized, Role: FOLLOWER
I20260809 23:09:51.734030 26731 ts_tablet_manager.cc:1397] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:09:51.734153 26732 consensus_queue.cc:260] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.734453 26732 ts_tablet_manager.cc:1428] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 23:09:51.734493 26487 heartbeater.cc:503] Master 127.8.141.254:36607 was elected leader, sending a full tablet report...
I20260809 23:09:51.735947 26731 raft_consensus.cc:348] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.736088 26731 raft_consensus.cc:374] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:09:51.736110 26731 raft_consensus.cc:729] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd73327b96c7474a9779c84aaa52be57, State: Initialized, Role: FOLLOWER
I20260809 23:09:51.736225 26731 consensus_queue.cc:260] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.736548 26731 ts_tablet_manager.cc:1428] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:09:51.736573 26265 heartbeater.cc:503] Master 127.8.141.254:36607 was elected leader, sending a full tablet report...
W20260809 23:09:51.762138 26488 tablet.cc:2369] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:09:51.782739 26739 raft_consensus.cc:482] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:09:51.782886 26739 raft_consensus.cc:504] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.783360 26739 leader_election.cc:288] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b32c25c2d8ba412789415038f0353e1c (127.8.141.195:34721), a33eb9386c84459499f5c15ea06f6ff2 (127.8.141.194:37777)
I20260809 23:09:51.787115 26270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.783637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:34721 (local address 127.8.141.193:48143)
0809 23:09:51.783705 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.783718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.783764 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.784145 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.784147 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.784150 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:51.784236 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.784237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.785214 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.785215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.786140 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.786143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.786793 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.786794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.786884 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.786888 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.786969 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.787026 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 23:09:51.787137 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.783720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:48143 (local address 127.8.141.195:34721)
0809 23:09:51.783771 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.783774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.783789 (+ 15us) server_negotiation.cc:407] Connection header received
0809 23:09:51.783822 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.783824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.783834 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.784055 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:51.784389 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.785129 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.786236 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.786692 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.786753 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.786761 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.787004 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.787054 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:09:51.787360 26726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.783894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:39199 (local address 127.8.141.194:37777)
0809 23:09:51.783957 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.783959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.784145 (+ 186us) server_negotiation.cc:407] Connection header received
0809 23:09:51.784232 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.784234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.784245 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.784312 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:09:51.784748 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.785253 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.786423 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.786889 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.786996 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.787006 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.787201 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.787287 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:09:51.787343 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.783783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37777 (local address 127.8.141.193:39199)
0809 23:09:51.784039 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.784050 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:09:51.784116 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:09:51.784390 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:09:51.784391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:09:51.784394 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:09:51.784652 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.784656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.785362 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.785363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.786297 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:09:51.786300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.787009 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.787010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:09:51.787095 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.787100 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:09:51.787183 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 23:09:51.787231 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":142,"threads_started":1}
I20260809 23:09:51.787601 26624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb6a5bbce54474fbab8b9f6a32a06e7" candidate_uuid: "cd73327b96c7474a9779c84aaa52be57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c25c2d8ba412789415038f0353e1c" is_pre_election: true
I20260809 23:09:51.787801 26412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb6a5bbce54474fbab8b9f6a32a06e7" candidate_uuid: "cd73327b96c7474a9779c84aaa52be57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a33eb9386c84459499f5c15ea06f6ff2" is_pre_election: true
I20260809 23:09:51.787849 26624 raft_consensus.cc:2454] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd73327b96c7474a9779c84aaa52be57 in term 0.
I20260809 23:09:51.787948 26412 raft_consensus.cc:2454] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd73327b96c7474a9779c84aaa52be57 in term 0.
I20260809 23:09:51.788163 26072 leader_election.cc:302] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32c25c2d8ba412789415038f0353e1c, cd73327b96c7474a9779c84aaa52be57; no voters:
I20260809 23:09:51.788383 26739 raft_consensus.cc:2781] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:09:51.788498 26739 raft_consensus.cc:482] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:09:51.788727 26739 raft_consensus.cc:3037] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:51.789690 26739 raft_consensus.cc:504] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.789872 26739 leader_election.cc:288] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [CANDIDATE]: Term 1 election: Requested vote from peers b32c25c2d8ba412789415038f0353e1c (127.8.141.195:34721), a33eb9386c84459499f5c15ea06f6ff2 (127.8.141.194:37777)
I20260809 23:09:51.790126 26624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb6a5bbce54474fbab8b9f6a32a06e7" candidate_uuid: "cd73327b96c7474a9779c84aaa52be57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b32c25c2d8ba412789415038f0353e1c"
I20260809 23:09:51.790185 26412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb6a5bbce54474fbab8b9f6a32a06e7" candidate_uuid: "cd73327b96c7474a9779c84aaa52be57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a33eb9386c84459499f5c15ea06f6ff2"
I20260809 23:09:51.790247 26624 raft_consensus.cc:3037] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:51.790310 26412 raft_consensus.cc:3037] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:09:51.791095 26624 raft_consensus.cc:2454] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd73327b96c7474a9779c84aaa52be57 in term 1.
I20260809 23:09:51.791114 26412 raft_consensus.cc:2454] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd73327b96c7474a9779c84aaa52be57 in term 1.
I20260809 23:09:51.791297 26072 leader_election.cc:302] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a33eb9386c84459499f5c15ea06f6ff2, cd73327b96c7474a9779c84aaa52be57; no voters:
I20260809 23:09:51.791416 26739 raft_consensus.cc:2781] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:09:51.791656 26739 raft_consensus.cc:686] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 1 LEADER]: Becoming Leader. State: Replica: cd73327b96c7474a9779c84aaa52be57, State: Running, Role: LEADER
I20260809 23:09:51.791790 26739 consensus_queue.cc:237] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } }
I20260809 23:09:51.792874 25936 catalog_manager.cc:5495] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd73327b96c7474a9779c84aaa52be57 (127.8.141.193). New cstate: current_term: 1 leader_uuid: "cd73327b96c7474a9779c84aaa52be57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd73327b96c7474a9779c84aaa52be57" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 42253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 } health_report { overall_health: UNKNOWN } } }
I20260809 23:09:51.802932 26723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.800150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.8.141.193:42253)
0809 23:09:51.800220 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:09:51.800222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:09:51.800244 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:09:51.800284 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:09:51.800284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:09:51.800306 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:09:51.800395 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:09:51.800572 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.801069 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.801988 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:09:51.802224 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:09:51.802307 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:09:51.802513 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:09:51.802653 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:09:51.802752 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:09:51.802773 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 23:09:51.802794 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:09:51.804950 26624 raft_consensus.cc:1261] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 1 FOLLOWER]: Refusing update from remote peer cd73327b96c7474a9779c84aaa52be57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:51.805123 26412 raft_consensus.cc:1261] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 1 FOLLOWER]: Refusing update from remote peer cd73327b96c7474a9779c84aaa52be57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:09:51.805390 26742 consensus_queue.cc:1035] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32c25c2d8ba412789415038f0353e1c" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 34721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:51.805544 26739 consensus_queue.cc:1035] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a33eb9386c84459499f5c15ea06f6ff2" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:09:51.808040 26749 mvcc.cc:205] Tried to move back new op lower bound from 7316754398430187520 to 7316754398379945984. Current Snapshot: MvccSnapshot[applied={T|T < 7316754398430187520}]
I20260809 23:09:51.808290 26751 mvcc.cc:205] Tried to move back new op lower bound from 7316754398430187520 to 7316754398379945984. Current Snapshot: MvccSnapshot[applied={T|T < 7316754398430187520}]
I20260809 23:09:51.808545 26750 mvcc.cc:205] Tried to move back new op lower bound from 7316754398430187520 to 7316754398379945984. Current Snapshot: MvccSnapshot[applied={T|T < 7316754398430187520}]
I20260809 23:09:51.811090 25935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:51.811188 25935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:09:51.812047 25935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02f71332736f43c48fee489ca654527f: Sending DeleteTablet for 3 replicas of tablet 7eb6a5bbce54474fbab8b9f6a32a06e7
I20260809 23:09:51.812433 26596 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:51 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:36168
I20260809 23:09:51.812476 26385 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:51 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:53966
I20260809 23:09:51.812737 26761 tablet_replica.cc:330] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: stopping tablet replica
I20260809 23:09:51.812728 26161 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:09:51 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:37194
I20260809 23:09:51.812891 26761 raft_consensus.cc:2229] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:51.813009 26763 tablet_replica.cc:330] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57: stopping tablet replica
I20260809 23:09:51.813118 26763 raft_consensus.cc:2229] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P cd73327b96c7474a9779c84aaa52be57 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:09:51.813130 26761 raft_consensus.cc:2258] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:51.813212 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 26048
I20260809 23:09:51.816993 26760 tablet_replica.cc:330] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: stopping tablet replica
I20260809 23:09:51.817152 26760 raft_consensus.cc:2229] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:09:51.817268 26760 raft_consensus.cc:2258] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:09:51.822264 26760 ts_tablet_manager.cc:1905] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:51.824795 26760 ts_tablet_manager.cc:1918] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:51.824846 26761 ts_tablet_manager.cc:1905] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:09:51.824914 26760 log.cc:1196] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/7eb6a5bbce54474fbab8b9f6a32a06e7
I20260809 23:09:51.825219 26760 ts_tablet_manager.cc:1939] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P b32c25c2d8ba412789415038f0353e1c: Deleting consensus metadata
I20260809 23:09:51.825954 25915 catalog_manager.cc:4841] TS b32c25c2d8ba412789415038f0353e1c (127.8.141.195:34721): tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 (table test-table [id=febcec42cca7434d95ed5e6938eb63d4]) successfully deleted
I20260809 23:09:51.827700 26761 ts_tablet_manager.cc:1918] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:09:51.827813 26761 log.cc:1196] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/7eb6a5bbce54474fbab8b9f6a32a06e7
I20260809 23:09:51.828177 26761 ts_tablet_manager.cc:1939] T 7eb6a5bbce54474fbab8b9f6a32a06e7 P a33eb9386c84459499f5c15ea06f6ff2: Deleting consensus metadata
I20260809 23:09:51.828771 25915 catalog_manager.cc:4841] TS a33eb9386c84459499f5c15ea06f6ff2 (127.8.141.194:37777): tablet 7eb6a5bbce54474fbab8b9f6a32a06e7 (table test-table [id=febcec42cca7434d95ed5e6938eb63d4]) successfully deleted
W20260809 23:09:51.832713 25917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:42253 (error 108)
I20260809 23:09:51.832988 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 26277
I20260809 23:09:51.836850 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:09:51.836653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:42253 (local address 127.0.0.1:37208)
0809 23:09:51.836734 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 23:09:51.836795 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:42253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 23:09:51.843963 25917 catalog_manager.cc:4584] TS cd73327b96c7474a9779c84aaa52be57 (127.8.141.193:42253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7eb6a5bbce54474fbab8b9f6a32a06e7: Network error: Client connection negotiation failed: client connection to 127.8.141.193:42253: connect: Connection refused (error 111)
I20260809 23:09:51.849584 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 26493
I20260809 23:09:51.868500 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 25890
2026-08-09T23:09:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2359 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[26767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[26767](info): set up 2 sockets
Aug 09 23:09:51 dist-test-slave-j2vl krb5kdc[26767](info): commencing operation
krb5kdc: starting...
W20260809 23:09:53.253109 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:09:53 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786316993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:09:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:09:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/rangeradmin_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/rangeradmin_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/rangerlookup_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/rangerlookup_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
I20260809 23:09:53.457599 8759 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1V3VFd/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/postgres -l logfile start
2026-08-09 23:09:56.745 UTC [26822] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 23:09:56.746 UTC [26822] LOG: listening on IPv4 address "127.8.141.212", port 36745
2026-08-09 23:09:56.750 UTC [26822] LOG: listening on Unix socket "/tmp/.s.PGSQL.36745"
2026-08-09 23:09:56.779 UTC [26828] LOG: database system was shut down at 2026-08-09 23:09:54 UTC
2026-08-09 23:09:56.783 UTC [26822] LOG: database system is ready to accept connections
I20260809 23:09:58.033243 8759 mini_postgres.cc:96] Postgres bound to 36745
2026-08-09 23:09:58.042 UTC [26840] FATAL: database "slave" does not exist
127.8.141.212:36745 - accepting connections
I20260809 23:09:58.043584 8759 mini_ranger.cc:161] Starting Ranger...
I20260809 23:09:58.061764 8759 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 23:09:58.752708 8759 mini_ranger.cc:87] Created ranger Postgres database
I20260809 23:09:58.752795 8759 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-admin
2026-08-09 23:09:59,124 [I] DB FLAVOR :POSTGRES
2026-08-09 23:09:59,124 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:09:59,124 [I] Checking connection..
2026-08-09 23:09:59,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 23:09:59,440 [I] Checking connection passed.
2026-08-09 23:09:59,440 [I] --------- Verifying version history table ---------
2026-08-09 23:09:59,440 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:09:59,797 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:09:59,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:10:00,067 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:10:00,068 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 23:10:00,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 23:10:00.346 UTC [26953] WARNING: there is no transaction in progress
2026-08-09 23:10:00,357 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 23:10:00,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:10:00,640 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 23:10:00,640 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:10:00,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 23:10:00,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 23:10:01,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j2vl.c.gcp-upstream.internal','N') ;"
2026-08-09 23:10:01,560 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:10:01,871 [I] Table x_portal_user does not exist in database ranger
2026-08-09 23:10:01,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:10:02,143 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 23:10:02,143 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:10:02,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:10:02,715 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 23:10:02,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 23:10:03.910 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:03.930 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:03.950 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:03.969 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:03.989 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.024 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.098 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.106 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.126 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.141 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.155 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.174 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.185 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.195 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.204 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:04.214 UTC [27122] WARNING: there is no transaction in progress
2026-08-09 23:10:05,068 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 23:10:05,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 23:10:05,317 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 23:10:05,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:10:05,611 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 23:10:05,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:10:05,896 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 23:10:05,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:10:06,191 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:10:06,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j2vl.c.gcp-upstream.internal';"
2026-08-09 23:10:06,931 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 23:10:06,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 23:10:07,326 [I] DB_PATCHES have already been applied
I20260809 23:10:07.336748 8759 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-admin:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1V3VFd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 23:10:07.336867 8759 mini_ranger.cc:193] Using host: 127.8.141.212
I20260809 23:10:07.343137 8759 mini_ranger.cc:227] Ranger admin URL: http://127.8.141.212:59763
Aug 09, 2026 11:10:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:10:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 11:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 11:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 11:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 11:10:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 23:10:09.291998 8759 mini_ranger.cc:160] Time spent starting Ranger: real 11.248s user 0.000s sys 0.015s
Aug 09, 2026 11:10:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 11:10:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 11:10:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 11:10:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 11:10:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 23:10:32.122745 8759 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/rangerkms_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/rangerkms_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.141.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.141.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 23:10:32 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317032, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 23:10:32.219313 8759 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 23:10:32.231842 8759 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 23:10:33.389424 8759 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 23:10:33.389526 8759 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms
I20260809 23:10:33.389549 8759 mini_ranger_kms.cc:227] Using postgres at 127.8.141.212:36745
2026-08-09 23:10:33,693 [I] DB FLAVOR :POSTGRES
2026-08-09 23:10:33,693 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:10:33,693 [I] Checking connection
2026-08-09 23:10:33,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 23:10:33,980 [I] connection success
2026-08-09 23:10:33,981 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 23:10:33,981 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 23:10:33,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 23:10:34,253 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 23:10:34,253 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:10:34,253 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 23:10:34,254 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:36745/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 23:10:34,577 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 23:10:35.166165 8759 mini_ranger_kms.cc:310] Created kms service
I20260809 23:10:35.404601 8759 mini_ranger_kms.cc:326] Created kudu user
I20260809 23:10:35.459523 8759 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 23:10:35 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317032, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:10:35.917924 8759 mini_ranger_kms.cc:384] Added ranger policy
I20260809 23:10:35.918063 8759 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms:/tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 23:10:35.918087 8759 mini_ranger_kms.cc:240] Using host: 127.8.141.212
I20260809 23:10:35.923509 8759 mini_ranger_kms.cc:276] Ranger KMS PID: 27433
I20260809 23:10:35.923695 8759 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.141.212:36117
Aug 09, 2026 11:10:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:10:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:10:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 11:10:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 11:10:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 11:10:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 23:10:38 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317038, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.141.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 23:10:38 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317038, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.141.212@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:10:38.891144 8759 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.672s user 0.000s sys 0.284s
I20260809 23:10:38.891265 8759 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 23:10:38.891383 8759 mini_ranger_kms.cc:401] 127.8.141.212:36117/kms/v1/keys
Aug 09 23:10:39 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317038, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.141.212@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
Aug 09 23:10:41 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:10:41.785691 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44705
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:44705
--ranger_config_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36117/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:10:41.890486 27523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:10:41.890720 27523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:10:41.890743 27523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:10:41.892055 27523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:10:41.892130 27523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:10:41.892159 27523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:10:41.892192 27523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:10:41.892221 27523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:10:41.894027 27523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.141.212:36117/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:44705
--ranger_config_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:10:41.894323 27523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:10:41.894565 27523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:10:41.901145 27532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:10:41.901239 27523 server_base.cc:1034] running on GCE node
W20260809 23:10:41.901140 27536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:10:41.901417 27530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:10:41.901834 27523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:10:41.902225 27523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:10:41.903399 27523 hybrid_clock.cc:648] HybridClock initialized: now 1786317041903375 us; error 39 us; skew 500 ppm
Aug 09 23:10:41 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317041, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:10:41.908877 27523 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:10:41.909534 27523 webserver.cc:460] Webserver started at http://127.8.141.254:35407/ using document root <none> and password file <none>
I20260809 23:10:41.909767 27523 fs_manager.cc:359] Metadata directory not provided
I20260809 23:10:41.909830 27523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:10:41.909986 27523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:10:41 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317041, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:10:41.962618 27523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "7d74b6e2b4684701b9554d7cd855f376"
format_stamp: "Formatted at 2026-08-09 23:10:41 on dist-test-slave-j2vl"
server_key: "27c2e75cdd6930c620d48f14429619d5"
server_key_iv: "3d1bfa7affca70bfa315e485dc18a7d0"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:41.963100 27523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "7d74b6e2b4684701b9554d7cd855f376"
format_stamp: "Formatted at 2026-08-09 23:10:41 on dist-test-slave-j2vl"
server_key: "27c2e75cdd6930c620d48f14429619d5"
server_key_iv: "3d1bfa7affca70bfa315e485dc18a7d0"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:41.965104 27523 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260809 23:10:41.981599 27547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:41.981956 27523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:10:41.982088 27523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "7d74b6e2b4684701b9554d7cd855f376"
format_stamp: "Formatted at 2026-08-09 23:10:41 on dist-test-slave-j2vl"
server_key: "27c2e75cdd6930c620d48f14429619d5"
server_key_iv: "3d1bfa7affca70bfa315e485dc18a7d0"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:41.982208 27523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:10:41.998010 27523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:10:42.000411 27523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:10:42.000608 27523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:10:42.005935 27523 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:44705
I20260809 23:10:42.005983 27631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:44705 every 8 connection(s)
I20260809 23:10:42.006402 27523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:10:42.007427 27634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:10:42.010993 27634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376: Bootstrap starting.
I20260809 23:10:42.011909 27634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376: Neither blocks nor log segments found. Creating new log.
I20260809 23:10:42.012324 27634 log.cc:824] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376: Log is configured to *not* fsync() on all Append() calls
I20260809 23:10:42.013283 27634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376: No bootstrap required, opened a new log
I20260809 23:10:42.014187 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 27523
I20260809 23:10:42.014423 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:10:42.014663 8759 external_mini_cluster.cc:1428] Setting key 0de8cd76f7431aec0afea53e68bc33ff
I20260809 23:10:42.016649 27634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74b6e2b4684701b9554d7cd855f376" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44705 } }
I20260809 23:10:42.016906 27634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:10:42.016981 27634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d74b6e2b4684701b9554d7cd855f376, State: Initialized, Role: FOLLOWER
I20260809 23:10:42.017163 27634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74b6e2b4684701b9554d7cd855f376" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44705 } }
I20260809 23:10:42.017249 27634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:10:42.017280 27634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:10:42.017369 27634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:10:42.018340 27634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74b6e2b4684701b9554d7cd855f376" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44705 } }
I20260809 23:10:42.018515 27634 leader_election.cc:302] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d74b6e2b4684701b9554d7cd855f376; no voters:
I20260809 23:10:42.018750 27634 leader_election.cc:288] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:10:42.018803 27640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:10:42.019055 27640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [term 1 LEADER]: Becoming Leader. State: Replica: 7d74b6e2b4684701b9554d7cd855f376, State: Running, Role: LEADER
I20260809 23:10:42.019169 27634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:10:42.019192 27640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74b6e2b4684701b9554d7cd855f376" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44705 } }
I20260809 23:10:42.019585 27640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d74b6e2b4684701b9554d7cd855f376" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74b6e2b4684701b9554d7cd855f376" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44705 } } }
I20260809 23:10:42.019654 27640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [sys.catalog]: This master's current role is: LEADER
I20260809 23:10:42.019783 27640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d74b6e2b4684701b9554d7cd855f376. Latest consensus state: current_term: 1 leader_uuid: "7d74b6e2b4684701b9554d7cd855f376" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d74b6e2b4684701b9554d7cd855f376" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44705 } } }
I20260809 23:10:42.019838 27640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376 [sys.catalog]: This master's current role is: LEADER
I20260809 23:10:42.020085 27644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:10:42.020676 27644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:10:42.021178 27634 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 23:10:42.023492 27644 catalog_manager.cc:1347] Generated new cluster ID: f1444ba9063a40a49401c1bf0c6fad0c
I20260809 23:10:42.023550 27644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:10:42 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:10:42.028481 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:42.015259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.8.141.254:44705)
0809 23:10:42.015526 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:42.015537 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:42.015576 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:10:42.016623 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:42.016632 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:42.016697 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:42.016872 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:10:42.017823 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:42.019195 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:42.020381 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:42.020985 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:42.021478 (+ 493us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:42.024189 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:10:42.024194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:10:42.024194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:10:42.024207 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:10:42.026328 (+ 2121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:42.026903 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:42.026907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:42.026907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:42.027015 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:42.027207 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:42.027208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:42.027208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:42.027517 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:10:42.028029 (+ 512us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:42.028101 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 23:10:42.028227 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":128,"threads_started":1}
I20260809 23:10:42.069409 27644 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:10:42.070129 27644 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:10:42.073647 27644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376: Generated new TSK 0
I20260809 23:10:42.073891 27644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:10:42.854122 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36117/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44705
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:10:42.982026 27698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:10:42.982287 27698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:10:42.982331 27698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:10:42.984035 27698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:10:42.984149 27698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:10:42.984210 27698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:10:42.986167 27698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.141.212:36117/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:10:42.986477 27698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:10:42.986732 27698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:10:42.993304 27710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:10:42.993511 27712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:10:42.993328 27708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:10:42.993386 27698 server_base.cc:1034] running on GCE node
I20260809 23:10:42.993795 27698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:10:42.994097 27698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:10:42.995271 27698 hybrid_clock.cc:648] HybridClock initialized: now 1786317042995251 us; error 34 us; skew 500 ppm
Aug 09 23:10:42 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:10:42.998061 27698 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:10:42.998598 27698 webserver.cc:460] Webserver started at http://127.8.141.193:44753/ using document root <none> and password file <none>
I20260809 23:10:42.998858 27698 fs_manager.cc:359] Metadata directory not provided
I20260809 23:10:42.998929 27698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:10:42.999084 27698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:10:42 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:10:43.016181 27698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "4b7476f95f524754afbcf6635387099f"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "0388b2bcb29c9b94db86aeb99104824f"
server_key_iv: "c161fab4348b54d991405dadee362f55"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.017019 27698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "4b7476f95f524754afbcf6635387099f"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "0388b2bcb29c9b94db86aeb99104824f"
server_key_iv: "c161fab4348b54d991405dadee362f55"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.021715 27698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.001s
I20260809 23:10:43.040174 27724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.040431 27698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 23:10:43.040539 27698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "4b7476f95f524754afbcf6635387099f"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "0388b2bcb29c9b94db86aeb99104824f"
server_key_iv: "c161fab4348b54d991405dadee362f55"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.043174 27698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:10:43.064239 27698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:10:43.067567 27698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:10:43.067727 27698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:10:43.068030 27698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:10:43.068557 27698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:10:43.068616 27698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.068652 27698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:10:43.068704 27698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.076434 27698 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:45781
I20260809 23:10:43.076967 27698 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:10:43.078946 27910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:45781 every 8 connection(s)
I20260809 23:10:43.084209 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 27698
I20260809 23:10:43.084345 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:10:43.084578 8759 external_mini_cluster.cc:1428] Setting key 29a2989698b6b1bef1ac8493bb2ea865
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317042, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:10:43.089963 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.079686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:45553 (local address 127.8.141.254:44705)
0809 23:10:43.079844 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.079847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.080069 (+ 222us) server_negotiation.cc:407] Connection header received
0809 23:10:43.080615 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.080616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.080631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.080690 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:10:43.081307 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.081870 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.082870 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.083091 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.083168 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.085988 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:10:43.085989 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:10:43.085990 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:10:43.085999 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:10:43.088066 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.088669 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.088670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.088671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.088745 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.089048 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.089048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.089049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.089206 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:10:43.089255 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.089716 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.089823 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260809 23:10:43.090564 27916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.079664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44705 (local address 127.8.141.193:45553)
0809 23:10:43.079926 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.079971 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.080497 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.080773 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.080779 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.081005 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:10:43.081199 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.081203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.081973 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.081974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.082768 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.082770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.083176 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.083178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.083313 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.083883 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:10:43.083892 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:10:43.085730 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:10:43.088174 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:10:43.088185 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:10:43.088188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:10:43.088573 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:10:43.088823 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:10:43.088824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:10:43.088825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:10:43.088969 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:10:43.089302 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:10:43.089309 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:10:43.089534 (+ 225us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.090084 (+ 550us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.090271 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":121,"threads_started":1}
I20260809 23:10:43.091228 27912 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44705
I20260809 23:10:43.091331 27912 heartbeater.cc:464] Registering TS with master...
I20260809 23:10:43.091543 27912 heartbeater.cc:511] Master 127.8.141.254:44705 requested a full tablet report, sending...
I20260809 23:10:43.092154 27578 ts_manager.cc:194] Registered new tserver with Master: 4b7476f95f524754afbcf6635387099f (127.8.141.193:45781)
I20260809 23:10:43.092784 27578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:45553
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:10:43.121424 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36117/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44705
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:10:43.227447 27924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:10:43.227720 27924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:10:43.227768 27924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:10:43.229576 27924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:10:43.229665 27924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:10:43.229717 27924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:10:43.231710 27924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.141.212:36117/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:10:43.232039 27924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:10:43.232362 27924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 23:10:43.238361 27943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:10:43.238360 27938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:10:43.238549 27939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:10:43.239425 27924 server_base.cc:1034] running on GCE node
I20260809 23:10:43.239683 27924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:10:43.240056 27924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:10:43.241250 27924 hybrid_clock.cc:648] HybridClock initialized: now 1786317043241218 us; error 45 us; skew 500 ppm
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:10:43.244534 27924 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:10:43.245072 27924 webserver.cc:460] Webserver started at http://127.8.141.194:41251/ using document root <none> and password file <none>
I20260809 23:10:43.245421 27924 fs_manager.cc:359] Metadata directory not provided
I20260809 23:10:43.245532 27924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:10:43.245700 27924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:10:43.269268 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "d5a89d10a12d32f9a1be0088ab4ba00a"
server_key_iv: "908a7b9173eeb5f592d54a28494d4be6"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.269770 27924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "d5a89d10a12d32f9a1be0088ab4ba00a"
server_key_iv: "908a7b9173eeb5f592d54a28494d4be6"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.274776 27924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 23:10:43.301376 27961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.301627 27924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:10:43.301749 27924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "d5a89d10a12d32f9a1be0088ab4ba00a"
server_key_iv: "908a7b9173eeb5f592d54a28494d4be6"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.301851 27924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:10:43.321709 27924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:10:43.324326 27924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:10:43.324541 27924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:10:43.324841 27924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:10:43.325291 27924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:10:43.325376 27924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.325435 27924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:10:43.325587 27924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.334666 27924 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:42607
I20260809 23:10:43.334949 28144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:42607 every 8 connection(s)
I20260809 23:10:43.335147 27924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:10:43.342514 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 27924
I20260809 23:10:43.342659 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:10:43.342891 8759 external_mini_cluster.cc:1428] Setting key ff82b73a8b0718d38b942aa281618a20
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:10:43.348543 28148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.336228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44705 (local address 127.8.141.194:50705)
0809 23:10:43.336460 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.336518 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.337271 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.337728 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.337734 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.338152 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:10:43.338469 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.338475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.339540 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.339542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.340930 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.340933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.341560 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.341561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.341721 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.342397 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:10:43.342408 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:10:43.344286 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:10:43.346581 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:10:43.346590 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:10:43.346594 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:10:43.346934 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:10:43.347223 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:10:43.347224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:10:43.347225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:10:43.347389 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:10:43.347753 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:10:43.347760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:10:43.347945 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.348069 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.348255 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260809 23:10:43.348897 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.336508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:50705 (local address 127.8.141.254:44705)
0809 23:10:43.336575 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.336578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.336633 (+ 55us) server_negotiation.cc:407] Connection header received
0809 23:10:43.337518 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.337519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.337541 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.337623 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:10:43.338586 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.339422 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.341034 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.341408 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.341480 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.344478 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:10:43.344480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:10:43.344480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:10:43.344489 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:10:43.346463 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.347040 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.347041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.347042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.347118 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.347492 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.347493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.347493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.347653 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:10:43.347721 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.348703 (+ 982us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.348808 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:10:43.349686 28146 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44705
I20260809 23:10:43.349803 28146 heartbeater.cc:464] Registering TS with master...
I20260809 23:10:43.350193 28146 heartbeater.cc:511] Master 127.8.141.254:44705 requested a full tablet report, sending...
I20260809 23:10:43.350662 27578 ts_manager.cc:194] Registered new tserver with Master: f0a2edb58b8d45d48dc75a8ad1f9e2dd (127.8.141.194:42607)
I20260809 23:10:43.351166 27578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:50705
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:10:43.382426 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36117/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44705
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:32895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:10:43.510196 28154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:10:43.510443 28154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:10:43.510473 28154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:10:43.511929 28154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:10:43.511981 28154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:10:43.512012 28154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:10:43.513958 28154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:32895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.141.212:36117/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:44705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:10:43.514258 28154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:10:43.514533 28154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:10:43.520560 28164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:10:43.520576 28162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:10:43.520566 28161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:10:43.521202 28154 server_base.cc:1034] running on GCE node
I20260809 23:10:43.521518 28154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:10:43.521817 28154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:10:43.523032 28154 hybrid_clock.cc:648] HybridClock initialized: now 1786317043523016 us; error 35 us; skew 500 ppm
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:10:43.526319 28154 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:10:43.526880 28154 webserver.cc:460] Webserver started at http://127.8.141.195:38895/ using document root <none> and password file <none>
I20260809 23:10:43.527143 28154 fs_manager.cc:359] Metadata directory not provided
I20260809 23:10:43.527221 28154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:10:43.527388 28154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:10:43.539916 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "133e2e7f5dfe48c9a37e92136b202422"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "0e495985b279b3177e0ea6000d5708df"
server_key_iv: "72b06e8cbc3c16c0e961061649953d54"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.540549 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "133e2e7f5dfe48c9a37e92136b202422"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "0e495985b279b3177e0ea6000d5708df"
server_key_iv: "72b06e8cbc3c16c0e961061649953d54"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.553203 28154 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.002s sys 0.000s
I20260809 23:10:43.569254 28178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.569571 28154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 23:10:43.569671 28154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "133e2e7f5dfe48c9a37e92136b202422"
format_stamp: "Formatted at 2026-08-09 23:10:43 on dist-test-slave-j2vl"
server_key: "0e495985b279b3177e0ea6000d5708df"
server_key_iv: "72b06e8cbc3c16c0e961061649953d54"
server_key_version: "kuduclusterkey@0"
I20260809 23:10:43.569769 28154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:10:43.582927 28154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:10:43.586093 28154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:10:43.586373 28154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:10:43.586761 28154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:10:43.587299 28154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:10:43.587404 28154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.587484 28154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:10:43.587553 28154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:10:43.598964 28154 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:43093
I20260809 23:10:43.599519 28154 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:10:43.600051 28356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:43093 every 8 connection(s)
I20260809 23:10:43.602015 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 28154
I20260809 23:10:43.602144 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:10:43.602344 8759 external_mini_cluster.cc:1428] Setting key 246373af9853993d54248c2a277d22f5
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:10:43.612892 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.600401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:44013 (local address 127.8.141.254:44705)
0809 23:10:43.600468 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.600471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.601699 (+ 1228us) server_negotiation.cc:407] Connection header received
0809 23:10:43.602398 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.602399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.602414 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.602447 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:10:43.603436 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.604263 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.605864 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.606182 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.606231 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.609196 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:10:43.609198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:10:43.609198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:10:43.609207 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:10:43.611082 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.611665 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.611667 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.611667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.611733 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.612073 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.612074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.612075 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.612228 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:10:43.612274 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.612684 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.612779 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:10:43.614302 28363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.600407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:44705 (local address 127.8.141.195:44013)
0809 23:10:43.601509 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.601561 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.602257 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.602712 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.602716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.603011 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:10:43.603312 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.603319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.604388 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.604389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.605706 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.605709 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.606335 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.606336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.606492 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.607193 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:10:43.607205 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:10:43.609063 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:10:43.611197 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:10:43.611206 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:10:43.611210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:10:43.611564 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:10:43.611814 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:10:43.611815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:10:43.611815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:10:43.611982 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:10:43.612324 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:10:43.612331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:10:43.612506 (+ 175us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.613118 (+ 612us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.613305 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1050,"thread_start_us":107,"threads_started":1}
I20260809 23:10:43.614987 28358 heartbeater.cc:347] Connected to a master server at 127.8.141.254:44705
I20260809 23:10:43.615130 28358 heartbeater.cc:464] Registering TS with master...
I20260809 23:10:43.615376 28358 heartbeater.cc:511] Master 127.8.141.254:44705 requested a full tablet report, sending...
I20260809 23:10:43.615792 27578 ts_manager.cc:194] Registered new tserver with Master: 133e2e7f5dfe48c9a37e92136b202422 (127.8.141.195:43093)
I20260809 23:10:43.616293 27578 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:44013
I20260809 23:10:43.625136 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:10:43.638363 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.627087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.8.141.254:44705)
0809 23:10:43.627150 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.627153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.629657 (+ 2504us) server_negotiation.cc:407] Connection header received
0809 23:10:43.629704 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.629705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.629720 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.629805 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:10:43.630680 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.631452 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.632630 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.632937 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.633018 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.634784 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:10:43.634786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:10:43.634786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:10:43.634796 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:10:43.636698 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.637162 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.637164 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.637164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.637233 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:10:43.637508 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:10:43.637510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:10:43.637510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:10:43.637773 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:10:43.637897 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.638140 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.638245 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:10:43.641498 27578 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:10:43.644176 27578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:10:43.654475 28383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.648643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:43093 (local address 127.0.0.1:38662)
0809 23:10:43.649478 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.649489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.649551 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.649831 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.649832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.649834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:10:43.649883 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.649885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.651209 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.651210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.652734 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.652738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.653919 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.653920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.654009 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.654026 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.654101 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.654162 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":268,"thread_start_us":88,"threads_started":1}
I20260809 23:10:43.654991 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.649168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.8.141.193:45781)
0809 23:10:43.649317 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.649323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.649333 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:10:43.649402 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.649404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.649421 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.649699 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:10:43.649751 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.651225 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.652880 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.654344 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.654766 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.654850 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.654877 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.654921 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260809 23:10:43.655004 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.648979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38662 (local address 127.8.141.195:43093)
0809 23:10:43.649411 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.649417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.649612 (+ 195us) server_negotiation.cc:407] Connection header received
0809 23:10:43.649658 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.649660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.649676 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.649749 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:10:43.649967 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.651092 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.652871 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.653710 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.653789 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.653877 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.654858 (+ 981us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.654909 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":85,"threads_started":1}
I20260809 23:10:43.654675 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.648297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:45781 (local address 127.0.0.1:42298)
0809 23:10:43.648818 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.648827 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.648862 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.649538 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.649541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.649544 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:10:43.649626 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.649628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.651323 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.651324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.652750 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.652754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.654455 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.654456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.654524 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.654528 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.654557 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.654600 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":2,"thread_start_us":163,"threads_started":1}
I20260809 23:10:43.656200 28258 tablet_service.cc:1467] Processing CreateTablet for tablet 1767d6bba96f4e198f17d239db63e065 (DEFAULT_TABLE table=test-table [id=d35d0e44034b4606a9cabd53f87869a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:10:43.656611 28258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1767d6bba96f4e198f17d239db63e065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:10:43.659659 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.648292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:42607 (local address 127.0.0.1:51092)
0809 23:10:43.648612 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.648652 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.648721 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.655636 (+ 6915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.655638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.655641 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:10:43.655687 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.655689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.657134 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.657135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.658286 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.658291 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.659379 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.659380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.659469 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.659476 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.659514 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.659564 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":164,"threads_started":1}
I20260809 23:10:43.660166 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.648779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51092 (local address 127.8.141.194:42607)
0809 23:10:43.651541 (+ 2762us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.651546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.651559 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:10:43.651600 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.651602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.651618 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.655787 (+ 4169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:10:43.655907 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.657008 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.658438 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.659261 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.659893 (+ 632us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.660002 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.660033 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.660076 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2713,"thread_start_us":83,"threads_started":1}
I20260809 23:10:43.660887 27799 tablet_service.cc:1467] Processing CreateTablet for tablet 1767d6bba96f4e198f17d239db63e065 (DEFAULT_TABLE table=test-table [id=d35d0e44034b4606a9cabd53f87869a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:10:43.660889 28031 tablet_service.cc:1467] Processing CreateTablet for tablet 1767d6bba96f4e198f17d239db63e065 (DEFAULT_TABLE table=test-table [id=d35d0e44034b4606a9cabd53f87869a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:10:43.661249 28031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1767d6bba96f4e198f17d239db63e065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:10:43.661249 27799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1767d6bba96f4e198f17d239db63e065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:10:43.666255 28390 tablet_bootstrap.cc:492] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: Bootstrap starting.
I20260809 23:10:43.667078 28388 tablet_bootstrap.cc:492] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Bootstrap starting.
I20260809 23:10:43.667572 28390 tablet_bootstrap.cc:654] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: Neither blocks nor log segments found. Creating new log.
I20260809 23:10:43.668068 28390 log.cc:824] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:10:43.668323 28388 tablet_bootstrap.cc:654] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Neither blocks nor log segments found. Creating new log.
I20260809 23:10:43.668781 28388 log.cc:824] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Log is configured to *not* fsync() on all Append() calls
I20260809 23:10:43.669469 28392 tablet_bootstrap.cc:492] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f: Bootstrap starting.
I20260809 23:10:43.670311 28392 tablet_bootstrap.cc:654] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f: Neither blocks nor log segments found. Creating new log.
I20260809 23:10:43.670702 28392 log.cc:824] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:10:43.671622 28388 tablet_bootstrap.cc:492] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: No bootstrap required, opened a new log
I20260809 23:10:43.671746 28388 ts_tablet_manager.cc:1397] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 23:10:43.674460 28388 raft_consensus.cc:348] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.674682 28388 raft_consensus.cc:374] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:10:43.674732 28388 raft_consensus.cc:729] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 133e2e7f5dfe48c9a37e92136b202422, State: Initialized, Role: FOLLOWER
I20260809 23:10:43.674954 28388 consensus_queue.cc:260] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.675259 28388 ts_tablet_manager.cc:1428] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:10:43.675427 28358 heartbeater.cc:503] Master 127.8.141.254:44705 was elected leader, sending a full tablet report...
I20260809 23:10:43.676666 28392 tablet_bootstrap.cc:492] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f: No bootstrap required, opened a new log
I20260809 23:10:43.676752 28392 ts_tablet_manager.cc:1397] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 23:10:43.679841 28392 raft_consensus.cc:348] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.680051 28392 raft_consensus.cc:374] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:10:43.680083 28392 raft_consensus.cc:729] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7476f95f524754afbcf6635387099f, State: Initialized, Role: FOLLOWER
I20260809 23:10:43.680228 28392 consensus_queue.cc:260] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.680554 28392 ts_tablet_manager.cc:1428] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 23:10:43.680612 27912 heartbeater.cc:503] Master 127.8.141.254:44705 was elected leader, sending a full tablet report...
I20260809 23:10:43.682457 28390 tablet_bootstrap.cc:492] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: No bootstrap required, opened a new log
I20260809 23:10:43.682562 28390 ts_tablet_manager.cc:1397] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260809 23:10:43.685998 28390 raft_consensus.cc:348] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.686290 28390 raft_consensus.cc:374] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:10:43.686383 28390 raft_consensus.cc:729] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a2edb58b8d45d48dc75a8ad1f9e2dd, State: Initialized, Role: FOLLOWER
I20260809 23:10:43.686578 28390 consensus_queue.cc:260] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.687018 28390 ts_tablet_manager.cc:1428] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 23:10:43.687047 28146 heartbeater.cc:503] Master 127.8.141.254:44705 was elected leader, sending a full tablet report...
I20260809 23:10:43.716861 28398 raft_consensus.cc:482] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:10:43.717276 28398 raft_consensus.cc:504] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.718215 28398 leader_election.cc:288] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0a2edb58b8d45d48dc75a8ad1f9e2dd (127.8.141.194:42607), 133e2e7f5dfe48c9a37e92136b202422 (127.8.141.195:43093)
I20260809 23:10:43.722309 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.718140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:35291 (local address 127.8.141.195:43093)
0809 23:10:43.718188 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.718191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.718581 (+ 390us) server_negotiation.cc:407] Connection header received
0809 23:10:43.718655 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.718656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.718670 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.718730 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:10:43.718995 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.719767 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.721334 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.721922 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.721971 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.721981 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.722194 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.722235 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:10:43.722733 28400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.717891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:43093 (local address 127.8.141.193:35291)
0809 23:10:43.718486 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.718499 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.718593 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.718790 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.718792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.718794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:10:43.718914 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.718915 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.719928 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.719929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.721231 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.721234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.722041 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.722042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.722117 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.722121 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.722586 (+ 465us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.722635 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":133,"threads_started":1}
I20260809 23:10:43.723237 28285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1767d6bba96f4e198f17d239db63e065" candidate_uuid: "4b7476f95f524754afbcf6635387099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "133e2e7f5dfe48c9a37e92136b202422" is_pre_election: true
I20260809 23:10:43.723430 28285 raft_consensus.cc:2454] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b7476f95f524754afbcf6635387099f in term 0.
I20260809 23:10:43.723863 28384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.718237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:34537 (local address 127.8.141.194:42607)
0809 23:10:43.718281 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.718283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.718754 (+ 471us) server_negotiation.cc:407] Connection header received
0809 23:10:43.718887 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.718888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.718901 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.718932 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:10:43.719220 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.720224 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.721847 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.722614 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.722670 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.722679 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.723750 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.723798 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:10:43.723848 27726 leader_election.cc:302] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 133e2e7f5dfe48c9a37e92136b202422, 4b7476f95f524754afbcf6635387099f; no voters:
I20260809 23:10:43.723985 28398 raft_consensus.cc:2781] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:10:43.724053 28398 raft_consensus.cc:482] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:10:43.724079 28398 raft_consensus.cc:3037] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:10:43.725129 28398 raft_consensus.cc:504] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.725310 28398 leader_election.cc:288] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [CANDIDATE]: Term 1 election: Requested vote from peers f0a2edb58b8d45d48dc75a8ad1f9e2dd (127.8.141.194:42607), 133e2e7f5dfe48c9a37e92136b202422 (127.8.141.195:43093)
I20260809 23:10:43.725963 28285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1767d6bba96f4e198f17d239db63e065" candidate_uuid: "4b7476f95f524754afbcf6635387099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "133e2e7f5dfe48c9a37e92136b202422"
I20260809 23:10:43.726097 28285 raft_consensus.cc:3037] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:10:43.727479 28285 raft_consensus.cc:2454] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b7476f95f524754afbcf6635387099f in term 1.
I20260809 23:10:43.727763 27726 leader_election.cc:302] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 133e2e7f5dfe48c9a37e92136b202422, 4b7476f95f524754afbcf6635387099f; no voters:
I20260809 23:10:43.727949 28398 raft_consensus.cc:2781] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:10:43.728152 28398 raft_consensus.cc:686] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [term 1 LEADER]: Becoming Leader. State: Replica: 4b7476f95f524754afbcf6635387099f, State: Running, Role: LEADER
I20260809 23:10:43.728261 28398 consensus_queue.cc:237] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } }
I20260809 23:10:43.729393 27576 catalog_manager.cc:5495] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b7476f95f524754afbcf6635387099f (127.8.141.193). New cstate: current_term: 1 leader_uuid: "4b7476f95f524754afbcf6635387099f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7476f95f524754afbcf6635387099f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 } health_report { overall_health: UNKNOWN } } }
I20260809 23:10:43.733429 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.717904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:42607 (local address 127.8.141.193:34537)
0809 23:10:43.718699 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.718708 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:10:43.718777 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:10:43.719004 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:10:43.719005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:10:43.719022 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:10:43.719120 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.719122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.720502 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.720503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.721734 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:10:43.721738 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.722800 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.722801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:10:43.722874 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.722877 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:10:43.730681 (+ 7804us) client_negotiation.cc:240] Negotiation successful
0809 23:10:43.733245 (+ 2564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":192,"threads_started":1}
Aug 09 23:10:43 dist-test-slave-j2vl krb5kdc[26767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317043, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:10:43.734692 28068 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1767d6bba96f4e198f17d239db63e065" candidate_uuid: "4b7476f95f524754afbcf6635387099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" is_pre_election: true
I20260809 23:10:43.734928 28068 raft_consensus.cc:2454] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b7476f95f524754afbcf6635387099f in term 0.
I20260809 23:10:43.734710 28067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1767d6bba96f4e198f17d239db63e065" candidate_uuid: "4b7476f95f524754afbcf6635387099f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd"
I20260809 23:10:43.735323 28067 raft_consensus.cc:3037] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:10:43.736402 28067 raft_consensus.cc:2454] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b7476f95f524754afbcf6635387099f in term 1.
I20260809 23:10:43.738539 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.735016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.8.141.193:45781)
0809 23:10:43.735128 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 23:10:43.735130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:10:43.735141 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:10:43.735272 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:10:43.735273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:10:43.735292 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:10:43.735361 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:10:43.735570 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.736300 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.737426 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:10:43.737744 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:10:43.737829 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:10:43.738030 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:10:43.738169 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:10:43.738253 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:10:43.738327 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 23:10:43.738352 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 23:10:43.740893 28067 raft_consensus.cc:1261] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 1 FOLLOWER]: Refusing update from remote peer 4b7476f95f524754afbcf6635387099f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:10:43.740922 28285 raft_consensus.cc:1261] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 1 FOLLOWER]: Refusing update from remote peer 4b7476f95f524754afbcf6635387099f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:10:43.741222 28398 consensus_queue.cc:1035] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a2edb58b8d45d48dc75a8ad1f9e2dd" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 42607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:10:43.741752 28398 consensus_queue.cc:1035] T 1767d6bba96f4e198f17d239db63e065 P 4b7476f95f524754afbcf6635387099f [LEADER]: Connected to new peer: Peer: permanent_uuid: "133e2e7f5dfe48c9a37e92136b202422" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:10:43.761077 28407 mvcc.cc:205] Tried to move back new op lower bound from 7316754611159609344 to 7316754611112267776. Current Snapshot: MvccSnapshot[applied={T|T < 7316754611159609344}]
I20260809 23:10:43.765385 28412 mvcc.cc:205] Tried to move back new op lower bound from 7316754611159609344 to 7316754611112267776. Current Snapshot: MvccSnapshot[applied={T|T < 7316754611159609344}]
I20260809 23:10:43.768008 27576 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:10:43.768122 27576 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:10:43.769048 27576 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d74b6e2b4684701b9554d7cd855f376: Sending DeleteTablet for 3 replicas of tablet 1767d6bba96f4e198f17d239db63e065
I20260809 23:10:43.769562 27799 tablet_service.cc:1514] Processing DeleteTablet for tablet 1767d6bba96f4e198f17d239db63e065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:10:43 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:42298
I20260809 23:10:43.769560 28031 tablet_service.cc:1514] Processing DeleteTablet for tablet 1767d6bba96f4e198f17d239db63e065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:10:43 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:51092
I20260809 23:10:43.770035 28258 tablet_service.cc:1514] Processing DeleteTablet for tablet 1767d6bba96f4e198f17d239db63e065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:10:43 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:38662
I20260809 23:10:43.771010 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 27698
I20260809 23:10:43.774672 28426 tablet_replica.cc:330] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: stopping tablet replica
I20260809 23:10:43.774858 28426 raft_consensus.cc:2229] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:10:43.775008 28426 raft_consensus.cc:2258] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:10:43.779927 28426 ts_tablet_manager.cc:1905] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:10:43.783952 28426 ts_tablet_manager.cc:1918] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:10:43.784057 28426 log.cc:1196] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/1767d6bba96f4e198f17d239db63e065
I20260809 23:10:43.784845 28426 ts_tablet_manager.cc:1939] T 1767d6bba96f4e198f17d239db63e065 P 133e2e7f5dfe48c9a37e92136b202422: Deleting consensus metadata
I20260809 23:10:43.789901 28425 tablet_replica.cc:330] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: stopping tablet replica
I20260809 23:10:43.790087 28425 raft_consensus.cc:2229] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:10:43.790227 28425 raft_consensus.cc:2258] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:10:43.792797 27549 catalog_manager.cc:4841] TS 133e2e7f5dfe48c9a37e92136b202422 (127.8.141.195:43093): tablet 1767d6bba96f4e198f17d239db63e065 (table test-table [id=d35d0e44034b4606a9cabd53f87869a8]) successfully deleted
W20260809 23:10:43.805676 27549 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:45781 (error 108)
I20260809 23:10:43.806452 28425 ts_tablet_manager.cc:1905] T 1767d6bba96f4e198f17d239db63e065 P f0a2edb58b8d45d48dc75a8ad1f9e2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:10:43.806785 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 27924
I20260809 23:10:43.842120 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.841955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:45781 (local address 127.0.0.1:42322)
0809 23:10:43.842023 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.842069 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:45781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 23:10:43.842342 27549 catalog_manager.cc:4584] TS 4b7476f95f524754afbcf6635387099f (127.8.141.193:45781): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1767d6bba96f4e198f17d239db63e065: Network error: Client connection negotiation failed: client connection to 127.8.141.193:45781: connect: Connection refused (error 111)
I20260809 23:10:43.850981 28379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:10:43.850853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:42607 (local address 127.0.0.1:51094)
0809 23:10:43.850907 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:10:43.850933 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.194:42607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 23:10:43.851109 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 28154
I20260809 23:10:43.872560 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 27523
I20260809 23:10:43.890264 8759 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 23:10:44.247200 8759 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 23:10:44.247327 8759 mini_ranger.cc:66] Stopping Ranger...
I20260809 23:10:44.604959 8759 mini_ranger.cc:68] Stopped Ranger
2026-08-09 23:10:44.605 UTC [26822] LOG: received smart shutdown request
2026-08-09 23:10:44.610 UTC [26822] LOG: background worker "logical replication launcher" (PID 26835) exited with exit code 1
2026-08-09 23:10:44.610 UTC [26830] LOG: shutting down
2026-08-09 23:10:44.828 UTC [26822] LOG: database system is shut down
2026-08-09T23:10:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53078 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:10:45 dist-test-slave-j2vl krb5kdc[28443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:10:45 dist-test-slave-j2vl krb5kdc[28443](info): set up 2 sockets
Aug 09 23:10:45 dist-test-slave-j2vl krb5kdc[28443](info): commencing operation
krb5kdc: starting...
W20260809 23:10:46.267501 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:10:46 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:10:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/rangeradmin_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/rangeradmin_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/rangerlookup_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/rangerlookup_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
I20260809 23:10:46.472961 8759 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task1V3VFd/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/postgres -l logfile start
2026-08-09 23:10:49.505 UTC [28496] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 23:10:49.505 UTC [28496] LOG: listening on IPv4 address "127.8.141.212", port 57961
2026-08-09 23:10:49.510 UTC [28496] LOG: listening on Unix socket "/tmp/.s.PGSQL.57961"
2026-08-09 23:10:49.536 UTC [28506] LOG: database system was shut down at 2026-08-09 23:10:47 UTC
2026-08-09 23:10:49.542 UTC [28496] LOG: database system is ready to accept connections
I20260809 23:10:50.753475 8759 mini_postgres.cc:96] Postgres bound to 57961
2026-08-09 23:10:50.760 UTC [28515] FATAL: database "slave" does not exist
127.8.141.212:57961 - accepting connections
I20260809 23:10:50.761606 8759 mini_ranger.cc:161] Starting Ranger...
I20260809 23:10:50.780535 8759 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 23:10:51.419056 8759 mini_ranger.cc:87] Created ranger Postgres database
I20260809 23:10:51.419167 8759 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-admin
2026-08-09 23:10:51,741 [I] DB FLAVOR :POSTGRES
2026-08-09 23:10:51,742 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:10:51,742 [I] Checking connection..
2026-08-09 23:10:51,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 23:10:52,038 [I] Checking connection passed.
2026-08-09 23:10:52,038 [I] --------- Verifying version history table ---------
2026-08-09 23:10:52,039 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:10:52,313 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:10:52,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:10:52,606 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 23:10:52,606 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 23:10:52,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 23:10:52.885 UTC [28609] WARNING: there is no transaction in progress
2026-08-09 23:10:52,893 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 23:10:52,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 23:10:53,165 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 23:10:53,166 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:10:53,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 23:10:53,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 23:10:53,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-j2vl.c.gcp-upstream.internal','N') ;"
2026-08-09 23:10:54,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:10:54,276 [I] Table x_portal_user does not exist in database ranger
2026-08-09 23:10:54,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:10:54,566 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 23:10:54,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:10:54,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:10:55,135 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 23:10:55,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 23:10:56.399 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.428 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.452 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.476 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.505 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.535 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.611 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.621 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.648 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.666 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.680 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.698 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.712 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.721 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.733 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:56.744 UTC [28802] WARNING: there is no transaction in progress
2026-08-09 23:10:57,658 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 23:10:57,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 23:10:57,924 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 23:10:57,924 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 23:10:58,208 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 23:10:58,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 23:10:58,486 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 23:10:58,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 23:10:58,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 23:10:59,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-j2vl.c.gcp-upstream.internal';"
2026-08-09 23:10:59,326 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 23:10:59,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 23:10:59,596 [I] DB_PATCHES have already been applied
I20260809 23:10:59.605237 8759 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-admin:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task1V3VFd/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 23:10:59.605321 8759 mini_ranger.cc:193] Using host: 127.8.141.212
I20260809 23:10:59.609886 8759 mini_ranger.cc:227] Ranger admin URL: http://127.8.141.212:45275
Aug 09, 2026 11:10:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:10:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:11:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 11:11:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 11:11:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 11:11:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 11:11:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 23:11:01.026845 8759 mini_ranger.cc:160] Time spent starting Ranger: real 10.265s user 0.003s sys 0.010s
Aug 09, 2026 11:11:23 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 11:11:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 11:11:23 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 11:11:24 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 11:11:25 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 23:11:25.722445 8759 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.141.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.141.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/rangerkms_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.141.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/rangerkms_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.141.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.141.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.141.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/HTTP_127.8.141.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 23:11:25 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317085, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 23:11:25.834309 8759 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 23:11:25.850756 8759 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 23:11:27.173954 8759 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 23:11:27.174062 8759 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms
I20260809 23:11:27.174089 8759 mini_ranger_kms.cc:227] Using postgres at 127.8.141.212:57961
2026-08-09 23:11:27,378 [I] DB FLAVOR :POSTGRES
2026-08-09 23:11:27,379 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 23:11:27,379 [I] Checking connection
2026-08-09 23:11:27,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 23:11:27,652 [I] connection success
2026-08-09 23:11:27,652 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 23:11:27,652 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 23:11:27,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 23:11:27,955 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 23:11:27,955 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 23:11:27,955 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 23:11:27,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.141.212:57961/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 23:11:28,247 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 23:11:28.845662 8759 mini_ranger_kms.cc:310] Created kms service
I20260809 23:11:29.084858 8759 mini_ranger_kms.cc:326] Created kudu user
I20260809 23:11:29.134349 8759 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 23:11:29 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317085, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:29.557304 8759 mini_ranger_kms.cc:384] Added ranger policy
I20260809 23:11:29.557466 8759 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms:/tmp/dist-test-task1V3VFd/build/release/bin/postgresql.jar:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task1V3VFd/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 23:11:29.557485 8759 mini_ranger_kms.cc:240] Using host: 127.8.141.212
I20260809 23:11:29.561826 8759 mini_ranger_kms.cc:276] Ranger KMS PID: 29083
I20260809 23:11:29.561916 8759 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.141.212:36917
Aug 09, 2026 11:11:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 11:11:30 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 11:11:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 11:11:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 11:11:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 11:11:31 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 23:11:31 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317091, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.141.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 23:11:32 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317091, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.141.212@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:32.444929 8759 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.611s user 0.010s sys 0.174s
I20260809 23:11:32.445031 8759 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 23:11:32.445078 8759 mini_ranger_kms.cc:401] 127.8.141.212:36917/kms/v1/keys
Aug 09 23:11:32 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317091, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.141.212@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
Aug 09 23:11:35 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:11:35.350324 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42919
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42919
--ranger_config_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:35.438086 29163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:35.438362 29163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:35.438405 29163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:35.439658 29163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:35.439738 29163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:35.439762 29163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:35.439783 29163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:35.439811 29163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:35.441593 29163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.141.212:36917/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42919
--ranger_config_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:35.441893 29163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:35.442150 29163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:35.448578 29168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:35.448589 29171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:35.448746 29163 server_base.cc:1034] running on GCE node
W20260809 23:11:35.448590 29173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:35.449412 29163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:35.449757 29163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:35.450964 29163 hybrid_clock.cc:648] HybridClock initialized: now 1786317095450934 us; error 45 us; skew 500 ppm
Aug 09 23:11:35 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317095, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:35.453871 29163 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:11:35.454351 29163 webserver.cc:460] Webserver started at http://127.8.141.254:45029/ using document root <none> and password file <none>
I20260809 23:11:35.454605 29163 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:35.454672 29163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:35.454867 29163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:11:35 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317095, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:35.524488 29163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "1597efbef8674bd5929b1a5bceaae78c"
format_stamp: "Formatted at 2026-08-09 23:11:35 on dist-test-slave-j2vl"
server_key: "6113b12f028809afe6d1d1af37f0e28c"
server_key_iv: "99fd9854383eeaf65f6b0dae17b6d0cd"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:35.524951 29163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "1597efbef8674bd5929b1a5bceaae78c"
format_stamp: "Formatted at 2026-08-09 23:11:35 on dist-test-slave-j2vl"
server_key: "6113b12f028809afe6d1d1af37f0e28c"
server_key_iv: "99fd9854383eeaf65f6b0dae17b6d0cd"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:35.526711 29163 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 23:11:35.544628 29184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:35.544860 29163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:35.544950 29163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "1597efbef8674bd5929b1a5bceaae78c"
format_stamp: "Formatted at 2026-08-09 23:11:35 on dist-test-slave-j2vl"
server_key: "6113b12f028809afe6d1d1af37f0e28c"
server_key_iv: "99fd9854383eeaf65f6b0dae17b6d0cd"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:35.545042 29163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:35.564819 29163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:35.567267 29163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:35.567480 29163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:35.573419 29163 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42919
I20260809 23:11:35.573531 29254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42919 every 8 connection(s)
I20260809 23:11:35.573917 29163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:11:35.574800 29255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:35.577507 29255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Bootstrap starting.
I20260809 23:11:35.578150 29255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:35.578485 29255 log.cc:824] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:35.578882 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 29163
I20260809 23:11:35.578990 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:11:35.579178 8759 external_mini_cluster.cc:1428] Setting key 4b399b0528a22385ccfbfb851ddac8a6
I20260809 23:11:35.579296 29255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: No bootstrap required, opened a new log
I20260809 23:11:35.581734 29255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:35.582027 29255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:35.582170 29255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1597efbef8674bd5929b1a5bceaae78c, State: Initialized, Role: FOLLOWER
I20260809 23:11:35.582410 29255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:35.582605 29255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:11:35.582855 29255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:11:35.582948 29255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:35.583714 29255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:35.583844 29255 leader_election.cc:302] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1597efbef8674bd5929b1a5bceaae78c; no voters:
I20260809 23:11:35.584043 29255 leader_election.cc:288] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:11:35.584070 29269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:11:35.584385 29269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 LEADER]: Becoming Leader. State: Replica: 1597efbef8674bd5929b1a5bceaae78c, State: Running, Role: LEADER
I20260809 23:11:35.584537 29269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:35.584937 29270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1597efbef8674bd5929b1a5bceaae78c. Latest consensus state: current_term: 1 leader_uuid: "1597efbef8674bd5929b1a5bceaae78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } } }
I20260809 23:11:35.585060 29270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:35.584931 29271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1597efbef8674bd5929b1a5bceaae78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } } }
I20260809 23:11:35.585292 29271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:35.585320 29274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:11:35.584421 29255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:35.587106 29255 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 09 23:11:35 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:35.590906 29274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:11:35.595877 29265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:35.579802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.8.141.254:42919)
0809 23:11:35.581492 (+ 1690us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:35.581503 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:35.581538 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:11:35.581704 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:35.581712 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:35.581772 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:35.581928 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:35.582789 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:35.584223 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:35.585384 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:35.585995 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:35.586135 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:35.589836 (+ 3701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:35.589898 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:35.589899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:35.590005 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:35.593373 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:35.594064 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:35.594068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:35.594068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:35.594181 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:35.594411 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:35.594412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:35.594412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:35.594796 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:35.594947 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:35.595464 (+ 517us) server_negotiation.cc:299] Negotiation successful
0809 23:11:35.595644 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1632,"thread_start_us":86,"threads_started":1}
I20260809 23:11:35.601516 29274 catalog_manager.cc:1347] Generated new cluster ID: 31a3428dd8c04ef888d9043e0c761b93
I20260809 23:11:35.601600 29274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:11:35.627884 29274 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:11:35.628788 29274 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:11:35.634912 29274 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Generated new TSK 0
I20260809 23:11:35.635275 29274 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:11:36.422326 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42919
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:36.545646 29335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:36.546011 29335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:36.546088 29335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:36.547384 29335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:36.547459 29335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:36.547508 29335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:11:36.549109 29335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.141.212:36917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:36.549464 29335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:36.549737 29335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:11:36.557134 29335 server_base.cc:1034] running on GCE node
W20260809 23:11:36.557161 29345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:36.557358 29347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:36.557377 29351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:36.557797 29335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:36.558142 29335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:36.559340 29335 hybrid_clock.cc:648] HybridClock initialized: now 1786317096559296 us; error 68 us; skew 500 ppm
Aug 09 23:11:36 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:36.562755 29335 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:11:36.563338 29335 webserver.cc:460] Webserver started at http://127.8.141.193:44171/ using document root <none> and password file <none>
I20260809 23:11:36.563596 29335 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:36.563642 29335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:36.563792 29335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:11:36 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:36.579200 29335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "1cce424fd6c6471ba252d05a0fc8b459"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "c190d25be6a1654ba7ddf21030c9d7b7"
server_key_iv: "4b05dda0b6d5c228686dabcc8704b3cf"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:36.579697 29335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "1cce424fd6c6471ba252d05a0fc8b459"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "c190d25be6a1654ba7ddf21030c9d7b7"
server_key_iv: "4b05dda0b6d5c228686dabcc8704b3cf"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:36.581817 29335 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 23:11:36.596441 29361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:36.596760 29335 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:36.596918 29335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "1cce424fd6c6471ba252d05a0fc8b459"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "c190d25be6a1654ba7ddf21030c9d7b7"
server_key_iv: "4b05dda0b6d5c228686dabcc8704b3cf"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:36.597052 29335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:36.623593 29335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:36.626260 29335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:36.626497 29335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:36.626840 29335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:36.627267 29335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:36.627360 29335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:36.627463 29335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:36.627535 29335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:36.635164 29335 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:40831
I20260809 23:11:36.635382 29548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:40831 every 8 connection(s)
I20260809 23:11:36.635712 29335 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:11:36.642187 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 29335
I20260809 23:11:36.642316 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:11:36.642525 8759 external_mini_cluster.cc:1428] Setting key ebbaf871cc8b4f618df7d83a1ae3fd9d
Aug 09 23:11:36 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:36.648789 29553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:36.636883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42919 (local address 127.8.141.193:54171)
0809 23:11:36.637517 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:36.637571 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:36.638395 (+ 824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:36.638805 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:36.638809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:36.639077 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:36.639296 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:36.639301 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.640398 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.640399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:36.641333 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:36.641336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.641857 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.641858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:36.642008 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:36.642711 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:36.642722 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:36.644506 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:36.646811 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:36.646821 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:36.646825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:36.647232 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:36.647487 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:36.647488 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:36.647488 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:36.647651 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:36.647989 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:36.647997 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:36.648190 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 23:11:36.648322 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 23:11:36.648518 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":97,"threads_started":1}
I20260809 23:11:36.649525 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:36.636905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:54171 (local address 127.8.141.254:42919)
0809 23:11:36.637051 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:36.637054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:36.637694 (+ 640us) server_negotiation.cc:407] Connection header received
0809 23:11:36.638558 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:36.638560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:36.638577 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:36.638666 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:36.639432 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.640264 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.641513 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.641725 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.641821 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:36.644636 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:36.644637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:36.644638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:36.644646 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:36.646699 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:36.647333 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:36.647335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:36.647335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:36.647410 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:36.647730 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:36.647731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:36.647731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:36.647896 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:36.647960 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:36.649262 (+ 1302us) server_negotiation.cc:299] Negotiation successful
0809 23:11:36.649420 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":62,"threads_started":1}
I20260809 23:11:36.650372 29550 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42919
I20260809 23:11:36.650487 29550 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:36.650705 29550 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:36.651398 29204 ts_manager.cc:194] Registered new tserver with Master: 1cce424fd6c6471ba252d05a0fc8b459 (127.8.141.193:40831)
I20260809 23:11:36.652376 29204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:54171
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:11:36.682678 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42919
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:11:36 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 23:11:36.807374 29559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:36.807739 29559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:36.807837 29559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:36.809675 29559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:36.809744 29559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:36.809792 29559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:11:36.812157 29559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.141.212:36917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:36.812907 29559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:36.813299 29559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:36.822566 29572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:36.822835 29575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:36.823942 29559 server_base.cc:1034] running on GCE node
W20260809 23:11:36.825529 29571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:36.825873 29559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:36.826200 29559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:36.827397 29559 hybrid_clock.cc:648] HybridClock initialized: now 1786317096827372 us; error 42 us; skew 500 ppm
Aug 09 23:11:36 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:36.831139 29559 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:11:36.831744 29559 webserver.cc:460] Webserver started at http://127.8.141.194:39275/ using document root <none> and password file <none>
I20260809 23:11:36.832032 29559 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:36.832085 29559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:36.832216 29559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:11:36 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:36.848671 29559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "fc04d455f1d84b46b86562f179838cd9"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "aae9e461f5357759638b73e543f955c6"
server_key_iv: "05027f658e3d2bec7434098eaf8aabc2"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:36.849116 29559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "fc04d455f1d84b46b86562f179838cd9"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "aae9e461f5357759638b73e543f955c6"
server_key_iv: "05027f658e3d2bec7434098eaf8aabc2"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:36.851327 29559 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 23:11:36.882620 29594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:36.882896 29559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:11:36.883011 29559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "fc04d455f1d84b46b86562f179838cd9"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "aae9e461f5357759638b73e543f955c6"
server_key_iv: "05027f658e3d2bec7434098eaf8aabc2"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:36.883109 29559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:36.915248 29559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:36.920027 29559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:36.920274 29559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:36.920607 29559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:36.921067 29559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:36.921172 29559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:36.921249 29559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:36.921314 29559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:36.928495 29559 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:34485
I20260809 23:11:36.931865 29797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:34485 every 8 connection(s)
I20260809 23:11:36.932090 29559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:11:36.934631 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 29559
I20260809 23:11:36.934773 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:11:36.934974 8759 external_mini_cluster.cc:1428] Setting key 80c3ce4bdf1f5d7349a159cf69d37fec
Aug 09 23:11:36 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:36.949180 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:36.933643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:40511 (local address 127.8.141.254:42919)
0809 23:11:36.933695 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:36.933698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:36.933902 (+ 204us) server_negotiation.cc:407] Connection header received
0809 23:11:36.938578 (+ 4676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:36.938579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:36.938599 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:36.938633 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:36.939619 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.940407 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.941995 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.942296 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.942349 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:36.945202 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:36.945204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:36.945204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:36.945214 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:36.947177 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:36.947745 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:36.947747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:36.947747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:36.947814 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:36.948277 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:36.948279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:36.948279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:36.948450 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:36.948503 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:36.948964 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 23:11:36.949078 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:11:36.950073 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:36.933298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42919 (local address 127.8.141.194:40511)
0809 23:11:36.933771 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:36.933818 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:36.938418 (+ 4600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:36.938865 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:36.938871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:36.939164 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:36.939500 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:36.939506 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.940540 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.940541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:36.941882 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:36.941885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:36.942470 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:36.942471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:36.942618 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:36.943283 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:36.943295 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:36.945077 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:36.947294 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:36.947305 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:36.947308 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:36.947646 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:36.947999 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:36.948000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:36.948001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:36.948168 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:36.948558 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:36.948566 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:36.948749 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 23:11:36.949584 (+ 835us) client_negotiation.cc:240] Negotiation successful
0809 23:11:36.949799 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":134,"threads_started":1}
I20260809 23:11:36.950882 29798 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42919
I20260809 23:11:36.950999 29798 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:36.951217 29798 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:36.951622 29204 ts_manager.cc:194] Registered new tserver with Master: fc04d455f1d84b46b86562f179838cd9 (127.8.141.194:34485)
I20260809 23:11:36.952147 29204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:40511
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:11:36.992161 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42919
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:37.131974 29808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:37.132296 29808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:37.132334 29808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:37.134068 29808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:37.134217 29808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:37.134315 29808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:11:37.136615 29808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.141.212:36917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:37.137456 29808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:37.137825 29808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:37.145283 29816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:37.145313 29818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:37.146281 29808 server_base.cc:1034] running on GCE node
W20260809 23:11:37.146529 29815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:37.146816 29808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:37.147119 29808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:37.154489 29808 hybrid_clock.cc:648] HybridClock initialized: now 1786317097154465 us; error 67 us; skew 500 ppm
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:37.158635 29808 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:11:37.159314 29808 webserver.cc:460] Webserver started at http://127.8.141.195:32807/ using document root <none> and password file <none>
I20260809 23:11:37.159642 29808 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:37.159752 29808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:37.159935 29808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:37.177841 29808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "6d4c1221b6d74d69a9159d486f5ed75f"
format_stamp: "Formatted at 2026-08-09 23:11:37 on dist-test-slave-j2vl"
server_key: "76975ecf2ba54ab7e70c7e460df3480e"
server_key_iv: "274e6cb8e8e04c37e9cca506614e557d"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:37.181715 29808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "6d4c1221b6d74d69a9159d486f5ed75f"
format_stamp: "Formatted at 2026-08-09 23:11:37 on dist-test-slave-j2vl"
server_key: "76975ecf2ba54ab7e70c7e460df3480e"
server_key_iv: "274e6cb8e8e04c37e9cca506614e557d"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:37.184686 29808 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 23:11:37.200981 29832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:37.201283 29808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:11:37.201416 29808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "6d4c1221b6d74d69a9159d486f5ed75f"
format_stamp: "Formatted at 2026-08-09 23:11:37 on dist-test-slave-j2vl"
server_key: "76975ecf2ba54ab7e70c7e460df3480e"
server_key_iv: "274e6cb8e8e04c37e9cca506614e557d"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:37.201535 29808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:37.235257 29808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:37.238250 29808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:37.238523 29808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:37.238925 29808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:37.239406 29808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:37.239655 29808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:37.239758 29808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:37.239823 29808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:37.247637 29808 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:41673
I20260809 23:11:37.248245 29808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:11:37.253304 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 29808
I20260809 23:11:37.253497 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:11:37.253720 8759 external_mini_cluster.cc:1428] Setting key 5cbd74e5018f609dcd26546c27d96224
I20260809 23:11:37.263043 30016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:41673 every 8 connection(s)
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:37.286780 29552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:37.261862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:32801 (local address 127.8.141.254:42919)
0809 23:11:37.261916 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:37.261918 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:37.264517 (+ 2599us) server_negotiation.cc:407] Connection header received
0809 23:11:37.267326 (+ 2809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:37.267328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:37.267343 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:37.267387 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:37.270228 (+ 2841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.271017 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.272915 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.273216 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.273268 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:37.282864 (+ 9596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:37.282866 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:37.282867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:37.282878 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:37.284762 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:37.285417 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:37.285419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:37.285419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:37.285486 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:37.285865 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:37.285866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:37.285867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:37.286023 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:37.286081 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:37.286552 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 23:11:37.286658 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:11:37.288350 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:37.261848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42919 (local address 127.8.141.195:32801)
0809 23:11:37.263644 (+ 1796us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:37.263693 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:37.267180 (+ 3487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:37.267598 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:37.267603 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:37.269718 (+ 2115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:37.270076 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:37.270086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.271475 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.271476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:37.272795 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:37.272798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.275941 (+ 3143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.275942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:37.276896 (+ 954us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:37.277804 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:37.277817 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:37.282708 (+ 4891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:37.284883 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:37.284893 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:37.284897 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:37.285264 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:37.285564 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:37.285565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:37.285566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:37.285777 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:37.286119 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:37.286126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:37.286360 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 23:11:37.287852 (+ 1492us) client_negotiation.cc:240] Negotiation successful
0809 23:11:37.288044 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1744,"thread_start_us":115,"threads_started":1}
I20260809 23:11:37.289031 30018 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42919
I20260809 23:11:37.289132 30018 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:37.289377 30018 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:37.289820 29204 ts_manager.cc:194] Registered new tserver with Master: 6d4c1221b6d74d69a9159d486f5ed75f (127.8.141.195:41673)
I20260809 23:11:37.290334 29204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:32801
I20260809 23:11:37.298703 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:11:37.299461 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 29335
I20260809 23:11:37.390106 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 29559
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317096, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:37.431677 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 29808
I20260809 23:11:37.466246 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 29163
I20260809 23:11:37.481282 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42919
--webserver_interface=127.8.141.254
--webserver_port=45029
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42919
--ranger_config_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:37.586766 30023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:37.587001 30023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:37.587026 30023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:37.588265 30023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:37.588310 30023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:37.588325 30023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:37.588338 30023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:37.588351 30023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:37.590021 30023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42919
--ranger_config_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=45029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:37.590229 30023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:37.590399 30023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:37.595916 30031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:37.595913 30035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:37.595913 30032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:37.596019 30023 server_base.cc:1034] running on GCE node
I20260809 23:11:37.596514 30023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:37.596900 30023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:37.598106 30023 hybrid_clock.cc:648] HybridClock initialized: now 1786317097598089 us; error 68 us; skew 500 ppm
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:37.601001 30023 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:11:37.601706 30023 webserver.cc:460] Webserver started at http://127.8.141.254:45029/ using document root <none> and password file <none>
I20260809 23:11:37.601986 30023 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:37.602059 30023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:37.640914 30023 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:11:37.641856 30045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:37.642175 30023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:37.642306 30023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "1597efbef8674bd5929b1a5bceaae78c"
format_stamp: "Formatted at 2026-08-09 23:11:35 on dist-test-slave-j2vl"
server_key: "6113b12f028809afe6d1d1af37f0e28c"
server_key_iv: "99fd9854383eeaf65f6b0dae17b6d0cd"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:37.642715 30023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:37.662336 30023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:37.664618 30023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:37.664816 30023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:37.669523 30023 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42919
I20260809 23:11:37.669566 30131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42919 every 8 connection(s)
I20260809 23:11:37.669973 30023 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:11:37.671442 30132 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:11:37.671444 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 30023
I20260809 23:11:37.671738 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:40831
--local_ip_for_outbound_sockets=127.8.141.193
--tserver_master_addrs=127.8.141.254:42919
--webserver_port=44171
--webserver_interface=127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:37.672535 30132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Bootstrap starting.
I20260809 23:11:37.681586 30132 log.cc:824] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:37.683701 30132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:11:37.684069 30132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Bootstrap complete.
I20260809 23:11:37.686842 30132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:37.687114 30132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1597efbef8674bd5929b1a5bceaae78c, State: Initialized, Role: FOLLOWER
I20260809 23:11:37.687307 30132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:37.687397 30132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:11:37.687464 30132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:11:37.687510 30132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:11:37.688815 30132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:37.688954 30132 leader_election.cc:302] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1597efbef8674bd5929b1a5bceaae78c; no voters:
I20260809 23:11:37.689190 30132 leader_election.cc:288] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:11:37.689240 30141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:11:37.689400 30141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [term 2 LEADER]: Becoming Leader. State: Replica: 1597efbef8674bd5929b1a5bceaae78c, State: Running, Role: LEADER
I20260809 23:11:37.689507 30141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } }
I20260809 23:11:37.689540 30132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:37.689815 30141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1597efbef8674bd5929b1a5bceaae78c. Latest consensus state: current_term: 2 leader_uuid: "1597efbef8674bd5929b1a5bceaae78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } } }
I20260809 23:11:37.689893 30141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:37.689819 30142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1597efbef8674bd5929b1a5bceaae78c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1597efbef8674bd5929b1a5bceaae78c" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42919 } } }
I20260809 23:11:37.689947 30142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:37.690253 30145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:11:37.690845 30145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:11:37.691015 30145 catalog_manager.cc:1259] Loaded cluster ID: 31a3428dd8c04ef888d9043e0c761b93
I20260809 23:11:37.691061 30145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:11:37.691401 30132 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 23:11:37.692247 30145 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:11:37.692453 30145 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Loaded TSK: 0
I20260809 23:11:37.692678 30145 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:11:37.788735 30136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:37.788986 30136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:37.789031 30136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:37.790753 30136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:37.790834 30136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:37.790881 30136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:11:37.793010 30136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:40831
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=44171
--tserver_master_addrs=127.8.141.254:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:37.793284 30136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:37.793594 30136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:37.801723 30192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:37.801740 30187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:37.802196 30189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:37.803059 30136 server_base.cc:1034] running on GCE node
I20260809 23:11:37.803313 30136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:37.803640 30136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:37.804829 30136 hybrid_clock.cc:648] HybridClock initialized: now 1786317097804804 us; error 56 us; skew 500 ppm
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:37.808560 30136 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:11:37.809186 30136 webserver.cc:460] Webserver started at http://127.8.141.193:44171/ using document root <none> and password file <none>
I20260809 23:11:37.809505 30136 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:37.809594 30136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:37.834640 30136 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:11:37.847079 30210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:37.847327 30136 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.000s sys 0.000s
I20260809 23:11:37.847432 30136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "1cce424fd6c6471ba252d05a0fc8b459"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "c190d25be6a1654ba7ddf21030c9d7b7"
server_key_iv: "4b05dda0b6d5c228686dabcc8704b3cf"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:37.847965 30136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:37.873996 30136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:37.878141 30136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:37.878340 30136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:37.878751 30136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:37.879179 30136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:37.879235 30136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:37.879276 30136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:37.879298 30136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:37.887305 30136 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:40831
I20260809 23:11:37.887382 30416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:40831 every 8 connection(s)
I20260809 23:11:37.887745 30136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:11:37.891126 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 30136
I20260809 23:11:37.891446 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:34485
--local_ip_for_outbound_sockets=127.8.141.194
--tserver_master_addrs=127.8.141.254:42919
--webserver_port=39275
--webserver_interface=127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:11:37 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317097, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:37.905611 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:37.888546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42919 (local address 127.8.141.193:33511)
0809 23:11:37.888811 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:37.888870 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:37.889560 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:37.890089 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:37.890093 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:37.890360 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:37.890652 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:37.890659 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.891953 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.891954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:37.892899 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:37.892903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.894049 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.894050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:37.894208 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:37.899512 (+ 5304us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:37.899530 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:37.900985 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:37.903477 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:37.903488 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:37.903491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:37.903842 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:37.904152 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:37.904153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:37.904154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:37.904339 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:37.904886 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:37.904894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:37.905061 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 23:11:37.905139 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 23:11:37.905292 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
I20260809 23:11:37.906296 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:37.888729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:33511 (local address 127.8.141.254:42919)
0809 23:11:37.888993 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:37.889001 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:37.889035 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:11:37.889792 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:37.889798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:37.889855 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:37.890001 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:37.890789 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.891784 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.893008 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:37.893907 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:37.894039 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:37.901209 (+ 7170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:37.901227 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:37.901228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:37.901245 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:37.903332 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:37.903965 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:37.903969 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:37.903969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:37.904070 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:37.904430 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:37.904432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:37.904432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:37.904786 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:37.905480 (+ 694us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:37.905570 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 23:11:37.906089 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":125,"threads_started":1}
I20260809 23:11:37.907284 30418 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42919
I20260809 23:11:37.907418 30418 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:37.907653 30418 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
W20260809 23:11:37.908771 30418 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42919 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:11:37.908869 30418 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:37.908955 30418 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:37.909375 30418 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:37.909462 30418 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
W20260809 23:11:37.909829 30418 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:11:38.005893 30425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:38.006184 30425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:38.006222 30425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:38.008235 30425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:38.008318 30425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:38.008365 30425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:11:38.010627 30425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:34485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=39275
--tserver_master_addrs=127.8.141.254:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:38.010921 30425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:38.011224 30425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:38.017866 30432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:38.017825 30425 server_base.cc:1034] running on GCE node
W20260809 23:11:38.017772 30433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:38.018388 30436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:38.018869 30425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:38.019207 30425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:38.020473 30425 hybrid_clock.cc:648] HybridClock initialized: now 1786317098020446 us; error 44 us; skew 500 ppm
Aug 09 23:11:38 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:38.023918 30425 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:11:38.024498 30425 webserver.cc:460] Webserver started at http://127.8.141.194:39275/ using document root <none> and password file <none>
I20260809 23:11:38.024804 30425 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:38.024879 30425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:11:38 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:38.045049 30425 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 23:11:38.046403 30448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:38.046695 30425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:38.046814 30425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "fc04d455f1d84b46b86562f179838cd9"
format_stamp: "Formatted at 2026-08-09 23:11:36 on dist-test-slave-j2vl"
server_key: "aae9e461f5357759638b73e543f955c6"
server_key_iv: "05027f658e3d2bec7434098eaf8aabc2"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:38.047326 30425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:38.069665 30425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:38.071902 30425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:38.072093 30425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:38.072909 30425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:38.073453 30425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:38.073518 30425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:38.073560 30425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:38.073587 30425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:38.082161 30425 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:34485
I20260809 23:11:38.082617 30425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:11:38.083308 30636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:34485 every 8 connection(s)
Aug 09 23:11:38 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:38.091356 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 30425
I20260809 23:11:38.091724 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.141.212:36917/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:41673
--local_ip_for_outbound_sockets=127.8.141.195
--tserver_master_addrs=127.8.141.254:42919
--webserver_port=32807
--webserver_interface=127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:33645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:38.094677 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:38.083868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:37487 (local address 127.8.141.254:42919)
0809 23:11:38.083938 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:38.083940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:38.083957 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:11:38.084401 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:38.084402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:38.084418 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:38.084477 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:38.085068 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.085837 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.087448 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.087645 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.087709 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:38.090581 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:38.090583 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:38.090584 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:38.090592 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:38.092799 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:38.093430 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:38.093431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:38.093432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:38.093502 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:38.093830 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:38.093831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:38.093831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:38.094006 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:38.094075 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:38.094485 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 23:11:38.094579 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 23:11:38.094920 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:38.083492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42919 (local address 127.8.141.194:37487)
0809 23:11:38.083684 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:38.083730 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:38.084272 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:38.084544 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:38.084548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:38.084753 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:38.084954 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:38.084958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.085968 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.085969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:38.087333 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:38.087337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.087780 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.087781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:38.087938 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:38.088674 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:38.088683 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:38.090469 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:38.092922 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:38.092932 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:38.092935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:38.093324 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:38.093594 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:38.093595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:38.093595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:38.093761 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:38.094104 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:38.094112 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:38.094291 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 23:11:38.094461 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 23:11:38.094631 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":118,"threads_started":1}
I20260809 23:11:38.095574 30640 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42919
I20260809 23:11:38.095698 30640 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:38.095924 30640 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
W20260809 23:11:38.096647 30640 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42919 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:11:38.096737 30640 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:38.096824 30640 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:38.097259 30640 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:38.097369 30640 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
W20260809 23:11:38.097759 30640 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:11:38.200438 30643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:38.200685 30643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:38.200727 30643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:38.202071 30643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:38.202145 30643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:38.202194 30643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:11:38.203822 30643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.141.212:36917/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:41673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=32807
--tserver_master_addrs=127.8.141.254:42919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:38.204128 30643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:38.204458 30643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:38.210095 30654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:38.210489 30643 server_base.cc:1034] running on GCE node
W20260809 23:11:38.210613 30652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:38.213532 30650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:38.213801 30643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:38.214118 30643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:38.215276 30643 hybrid_clock.cc:648] HybridClock initialized: now 1786317098215248 us; error 46 us; skew 500 ppm
Aug 09 23:11:38 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:38.218397 30643 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:11:38.218979 30643 webserver.cc:460] Webserver started at http://127.8.141.195:32807/ using document root <none> and password file <none>
I20260809 23:11:38.219306 30643 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:38.219406 30643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 23:11:38 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:38.241333 30643 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 23:11:38.242422 30672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:38.242619 30643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:38.242722 30643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "6d4c1221b6d74d69a9159d486f5ed75f"
format_stamp: "Formatted at 2026-08-09 23:11:37 on dist-test-slave-j2vl"
server_key: "76975ecf2ba54ab7e70c7e460df3480e"
server_key_iv: "274e6cb8e8e04c37e9cca506614e557d"
server_key_version: "kuduclusterkey@0"
I20260809 23:11:38.243227 30643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:38.255009 30643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:38.257956 30643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:38.258157 30643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:38.258487 30643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:38.259075 30643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:38.259155 30643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:38.259197 30643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:38.259238 30643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:38.268337 30643 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:41673
I20260809 23:11:38.268487 30859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:41673 every 8 connection(s)
I20260809 23:11:38.268821 30643 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:11:38.270102 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 30643
Aug 09 23:11:38 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:38.280733 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:38.270544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.8.141.254:42919)
0809 23:11:38.271411 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:38.271413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:38.271560 (+ 147us) server_negotiation.cc:407] Connection header received
0809 23:11:38.271664 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:38.271665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:38.271677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:38.271721 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:38.272540 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.273234 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.274368 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.274685 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.274749 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:38.276663 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:38.276665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:38.276665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:38.276673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:38.279230 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:38.279654 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:38.279655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:38.279656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:38.279712 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:38.279948 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:38.279949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:38.279949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:38.280105 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:38.280277 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:38.280519 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 23:11:38.280618 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":81,"threads_started":1}
I20260809 23:11:38.281574 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:38.269778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:34729 (local address 127.8.141.254:42919)
0809 23:11:38.269832 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:38.269835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:38.270740 (+ 905us) server_negotiation.cc:407] Connection header received
0809 23:11:38.271301 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:38.271302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:38.271316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:38.271383 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:38.272245 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.273051 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.274564 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.274857 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.274893 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:38.277796 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:38.277798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:38.277798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:38.277807 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:38.279537 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:38.280086 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:38.280087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:38.280087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:38.280137 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:38.280448 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:38.280449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:38.280450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:38.280570 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:38.280627 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:38.281373 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 23:11:38.281471 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:11:38.282555 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:38.269801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42919 (local address 127.8.141.195:34729)
0809 23:11:38.270401 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:38.270453 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:38.271180 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:38.271518 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:38.271522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:38.271808 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:38.272119 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:38.272126 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.273163 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.273164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:38.274477 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:38.274480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:38.275004 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:38.275005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:38.275155 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:38.275715 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:38.275727 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:38.277671 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:38.279628 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:38.279636 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:38.279639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:38.280004 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:38.280210 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:38.280211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:38.280211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:38.280366 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:38.280677 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:38.280684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:38.281169 (+ 485us) client_negotiation.cc:769] Sending connection context
0809 23:11:38.282194 (+ 1025us) client_negotiation.cc:240] Negotiation successful
0809 23:11:38.282376 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":131,"threads_started":1}
I20260809 23:11:38.283133 30860 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42919
I20260809 23:11:38.283272 30860 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:38.283476 30860 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
W20260809 23:11:38.284173 30860 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42919 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 23:11:38.284268 30860 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:38.284349 30860 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:38.284644 30860 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:38.284727 30860 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
W20260809 23:11:38.285007 30860 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 23:11:38 dist-test-slave-j2vl krb5kdc[28443](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:38.910054 30418 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:38.910235 30418 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:38.911742 30074 ts_manager.cc:194] Registered new tserver with Master: 1cce424fd6c6471ba252d05a0fc8b459 (127.8.141.193:40831)
I20260809 23:11:38.912781 30074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:33511
I20260809 23:11:39.097988 30640 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:39.098170 30640 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:39.098555 30074 ts_manager.cc:194] Registered new tserver with Master: fc04d455f1d84b46b86562f179838cd9 (127.8.141.194:34485)
I20260809 23:11:39.099133 30074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:37487
Aug 09 23:11:39 dist-test-slave-j2vl krb5kdc[28443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317098, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for HTTP/127.8.141.212@KRBTEST.COM
I20260809 23:11:39.285218 30860 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:39.285435 30860 heartbeater.cc:511] Master 127.8.141.254:42919 requested a full tablet report, sending...
I20260809 23:11:39.285774 30074 ts_manager.cc:194] Registered new tserver with Master: 6d4c1221b6d74d69a9159d486f5ed75f (127.8.141.195:41673)
I20260809 23:11:39.286301 30074 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:34729
I20260809 23:11:39.290342 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:11:39.301196 30900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.291940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37214 (local address 127.8.141.254:42919)
0809 23:11:39.292551 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:39.292553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:39.292561 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:11:39.292585 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:39.292586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:39.292599 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:39.292670 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:39.293518 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.294218 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.295228 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.295535 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.295603 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.297525 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:39.297527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:39.297527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:39.297535 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:39.299465 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:39.299890 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:39.299891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:39.299891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:39.299976 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:39.300202 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:39.300203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:39.300203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:39.300449 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:39.300548 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:39.301001 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 23:11:39.301094 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":99,"threads_started":1}
I20260809 23:11:39.303934 30074 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:11:39.306572 30074 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:11:39.326100 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.314563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:34485 (local address 127.0.0.1:60626)
0809 23:11:39.316996 (+ 2433us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:39.317006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:39.317055 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:39.317318 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:39.317322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:39.317324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:39.317422 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.317424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.321652 (+ 4228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.321653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.322837 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.322839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.325746 (+ 2907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.325747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.325846 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.325860 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 23:11:39.325951 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 23:11:39.326008 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2416,"thread_start_us":65,"threads_started":1}
I20260809 23:11:39.326953 30917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.316137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.8.141.195:41673)
0809 23:11:39.322815 (+ 6678us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:39.322821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:39.322835 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:11:39.322885 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:39.322887 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:39.322906 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:39.323006 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:39.323253 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.324279 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.325614 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.326344 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.326419 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.326479 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:39.326822 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 23:11:39.326879 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6629,"thread_start_us":57,"threads_started":1}
I20260809 23:11:39.326751 30909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.315980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41673 (local address 127.0.0.1:54384)
0809 23:11:39.320568 (+ 4588us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:39.320579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:39.320623 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:39.323123 (+ 2500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:39.323125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:39.323128 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:39.323173 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.323174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.324387 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.324388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.325512 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.325515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.326518 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.326520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.326583 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.326587 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:11:39.326634 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 23:11:39.326677 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4569,"thread_start_us":79,"threads_started":1}
I20260809 23:11:39.327629 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.314857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42326 (local address 127.8.141.193:40831)
0809 23:11:39.316643 (+ 1786us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:39.316648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:39.316659 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:11:39.316695 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:39.316697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:39.316714 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:39.316786 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:39.318245 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.319311 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.323820 (+ 4509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.324622 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.324700 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.324762 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:39.327485 (+ 2723us) server_negotiation.cc:299] Negotiation successful
0809 23:11:39.327546 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1747,"thread_start_us":64,"threads_started":1}
I20260809 23:11:39.328068 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.314998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60626 (local address 127.8.141.194:34485)
0809 23:11:39.316941 (+ 1943us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:39.316945 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:39.317105 (+ 160us) server_negotiation.cc:407] Connection header received
0809 23:11:39.317139 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:39.317141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:39.317157 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:39.317221 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:39.317610 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.321553 (+ 3943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.324798 (+ 3245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.325581 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.325657 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.325708 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:39.327933 (+ 2225us) server_negotiation.cc:299] Negotiation successful
0809 23:11:39.327988 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1908,"thread_start_us":58,"threads_started":1}
I20260809 23:11:39.327973 30749 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6839c4f507474398485b8244f97792 (DEFAULT_TABLE table=test-table [id=39f0c5e491be46368598a507457784a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:39.328387 30749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6839c4f507474398485b8244f97792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:39.329123 30529 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6839c4f507474398485b8244f97792 (DEFAULT_TABLE table=test-table [id=39f0c5e491be46368598a507457784a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:39.329478 30529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6839c4f507474398485b8244f97792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:39.326959 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.314321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:40831 (local address 127.0.0.1:42326)
0809 23:11:39.316388 (+ 2067us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:39.316438 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:39.316503 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:39.317497 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:39.317498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:39.317499 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:39.317541 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.317542 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.319411 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.319413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.323726 (+ 4313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.323729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.326700 (+ 2971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.326700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.326781 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.326785 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:11:39.326831 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 23:11:39.326886 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2025,"thread_start_us":87,"threads_started":1}
I20260809 23:11:39.333433 30921 tablet_bootstrap.cc:492] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Bootstrap starting.
I20260809 23:11:39.334050 30310 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6839c4f507474398485b8244f97792 (DEFAULT_TABLE table=test-table [id=39f0c5e491be46368598a507457784a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:39.334182 30921 tablet_bootstrap.cc:654] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:39.334416 30310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6839c4f507474398485b8244f97792. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:39.334621 30921 log.cc:824] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:39.335460 30921 tablet_bootstrap.cc:492] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: No bootstrap required, opened a new log
I20260809 23:11:39.335526 30921 ts_tablet_manager.cc:1397] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:11:39.338008 30921 raft_consensus.cc:348] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.338191 30921 raft_consensus.cc:374] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:39.338222 30921 raft_consensus.cc:729] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc04d455f1d84b46b86562f179838cd9, State: Initialized, Role: FOLLOWER
I20260809 23:11:39.338696 30921 consensus_queue.cc:260] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.339107 30921 ts_tablet_manager.cc:1428] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 23:11:39.339212 30640 heartbeater.cc:503] Master 127.8.141.254:42919 was elected leader, sending a full tablet report...
I20260809 23:11:39.339792 30920 tablet_bootstrap.cc:492] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Bootstrap starting.
I20260809 23:11:39.340782 30920 tablet_bootstrap.cc:654] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:39.341120 30920 log.cc:824] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:39.341524 30923 tablet_bootstrap.cc:492] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459: Bootstrap starting.
I20260809 23:11:39.341993 30920 tablet_bootstrap.cc:492] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: No bootstrap required, opened a new log
I20260809 23:11:39.342072 30920 ts_tablet_manager.cc:1397] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:11:39.342372 30923 tablet_bootstrap.cc:654] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:39.342777 30923 log.cc:824] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:39.344627 30920 raft_consensus.cc:348] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.344810 30920 raft_consensus.cc:374] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:39.344842 30920 raft_consensus.cc:729] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d4c1221b6d74d69a9159d486f5ed75f, State: Initialized, Role: FOLLOWER
I20260809 23:11:39.345184 30920 consensus_queue.cc:260] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.345597 30920 ts_tablet_manager.cc:1428] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 23:11:39.345620 30860 heartbeater.cc:503] Master 127.8.141.254:42919 was elected leader, sending a full tablet report...
I20260809 23:11:39.350234 30923 tablet_bootstrap.cc:492] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459: No bootstrap required, opened a new log
I20260809 23:11:39.350325 30923 ts_tablet_manager.cc:1397] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260809 23:11:39.353005 30923 raft_consensus.cc:348] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.353190 30923 raft_consensus.cc:374] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:39.353221 30923 raft_consensus.cc:729] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cce424fd6c6471ba252d05a0fc8b459, State: Initialized, Role: FOLLOWER
I20260809 23:11:39.353412 30923 consensus_queue.cc:260] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.353724 30923 ts_tablet_manager.cc:1428] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 23:11:39.353782 30418 heartbeater.cc:503] Master 127.8.141.254:42919 was elected leader, sending a full tablet report...
W20260809 23:11:39.389461 30419 tablet.cc:2369] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:11:39.444767 30928 raft_consensus.cc:482] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:39.444943 30928 raft_consensus.cc:504] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.445459 30928 leader_election.cc:288] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cce424fd6c6471ba252d05a0fc8b459 (127.8.141.193:40831), fc04d455f1d84b46b86562f179838cd9 (127.8.141.194:34485)
I20260809 23:11:39.449944 30913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.446125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:51273 (local address 127.8.141.193:40831)
0809 23:11:39.446250 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:39.446252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:39.446263 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:11:39.446328 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:39.446329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:39.446339 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:39.446403 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:39.446712 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.447471 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.448936 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.449527 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.449581 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.449590 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:39.449827 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 23:11:39.449870 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 23:11:39.450253 30914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.446042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:48505 (local address 127.8.141.194:34485)
0809 23:11:39.446210 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:39.446213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:39.446424 (+ 211us) server_negotiation.cc:407] Connection header received
0809 23:11:39.446528 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:39.446529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:39.446542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:39.446573 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:39.446794 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.447560 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.448991 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.449684 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.449737 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.449748 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:39.450121 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 23:11:39.450166 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260809 23:11:39.450603 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.445660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:40831 (local address 127.8.141.195:51273)
0809 23:11:39.446161 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:39.446175 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:39.446256 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:39.446521 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:39.446523 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:39.446525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:39.446622 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.446625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.447594 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.447595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.448832 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.448835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.449675 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.449676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.449744 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.449748 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:11:39.450294 (+ 546us) client_negotiation.cc:240] Negotiation successful
0809 23:11:39.450457 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":129,"threads_started":1}
I20260809 23:11:39.450742 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.445740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:34485 (local address 127.8.141.195:48505)
0809 23:11:39.446367 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:39.446376 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:39.446451 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:39.446640 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:39.446641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:39.446643 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:39.446714 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.446715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.447698 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.447699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.448884 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:39.448887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.449814 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.449814 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:39.450018 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.450022 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:11:39.450393 (+ 371us) client_negotiation.cc:240] Negotiation successful
0809 23:11:39.450443 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":13,"thread_start_us":129,"threads_started":1}
I20260809 23:11:39.451212 30347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6839c4f507474398485b8244f97792" candidate_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cce424fd6c6471ba252d05a0fc8b459" is_pre_election: true
I20260809 23:11:39.451215 30546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6839c4f507474398485b8244f97792" candidate_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc04d455f1d84b46b86562f179838cd9" is_pre_election: true
I20260809 23:11:39.451397 30546 raft_consensus.cc:2454] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d4c1221b6d74d69a9159d486f5ed75f in term 0.
I20260809 23:11:39.451398 30347 raft_consensus.cc:2454] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d4c1221b6d74d69a9159d486f5ed75f in term 0.
I20260809 23:11:39.451660 30677 leader_election.cc:302] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d4c1221b6d74d69a9159d486f5ed75f, fc04d455f1d84b46b86562f179838cd9; no voters:
I20260809 23:11:39.451910 30928 raft_consensus.cc:2781] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:11:39.451978 30928 raft_consensus.cc:482] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:11:39.452003 30928 raft_consensus.cc:3037] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:39.453068 30928 raft_consensus.cc:504] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.453220 30928 leader_election.cc:288] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [CANDIDATE]: Term 1 election: Requested vote from peers 1cce424fd6c6471ba252d05a0fc8b459 (127.8.141.193:40831), fc04d455f1d84b46b86562f179838cd9 (127.8.141.194:34485)
I20260809 23:11:39.453533 30546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6839c4f507474398485b8244f97792" candidate_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc04d455f1d84b46b86562f179838cd9"
I20260809 23:11:39.453511 30347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6839c4f507474398485b8244f97792" candidate_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cce424fd6c6471ba252d05a0fc8b459"
I20260809 23:11:39.453617 30546 raft_consensus.cc:3037] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:39.453624 30347 raft_consensus.cc:3037] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:39.454571 30546 raft_consensus.cc:2454] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d4c1221b6d74d69a9159d486f5ed75f in term 1.
I20260809 23:11:39.454646 30347 raft_consensus.cc:2454] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d4c1221b6d74d69a9159d486f5ed75f in term 1.
I20260809 23:11:39.454859 30677 leader_election.cc:302] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d4c1221b6d74d69a9159d486f5ed75f, fc04d455f1d84b46b86562f179838cd9; no voters:
I20260809 23:11:39.454994 30928 raft_consensus.cc:2781] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:11:39.455174 30928 raft_consensus.cc:686] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 1 LEADER]: Becoming Leader. State: Replica: 6d4c1221b6d74d69a9159d486f5ed75f, State: Running, Role: LEADER
I20260809 23:11:39.455302 30928 consensus_queue.cc:237] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } }
I20260809 23:11:39.456239 30074 catalog_manager.cc:5495] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d4c1221b6d74d69a9159d486f5ed75f (127.8.141.195). New cstate: current_term: 1 leader_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d4c1221b6d74d69a9159d486f5ed75f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41673 } health_report { overall_health: HEALTHY } } }
I20260809 23:11:39.467751 30917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.464617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.8.141.195:41673)
0809 23:11:39.464684 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:39.464686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:39.464695 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:11:39.464732 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:39.464733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:39.464758 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:39.464850 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:11:39.465022 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.465792 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.466845 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:39.467151 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:39.467227 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:39.467392 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:11:39.467519 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:11:39.467601 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:39.467654 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 23:11:39.467674 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:11:39.469887 30546 raft_consensus.cc:1261] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 1 FOLLOWER]: Refusing update from remote peer 6d4c1221b6d74d69a9159d486f5ed75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:39.469926 30347 raft_consensus.cc:1261] T 8a6839c4f507474398485b8244f97792 P 1cce424fd6c6471ba252d05a0fc8b459 [term 1 FOLLOWER]: Refusing update from remote peer 6d4c1221b6d74d69a9159d486f5ed75f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:39.470321 30932 consensus_queue.cc:1035] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cce424fd6c6471ba252d05a0fc8b459" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 40831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:11:39.470495 30928 consensus_queue.cc:1035] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc04d455f1d84b46b86562f179838cd9" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 34485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:11:39.473745 30940 mvcc.cc:205] Tried to move back new op lower bound from 7316754839425744896 to 7316754839369625600. Current Snapshot: MvccSnapshot[applied={T|T < 7316754839425744896}]
I20260809 23:11:39.475443 30946 mvcc.cc:205] Tried to move back new op lower bound from 7316754839425744896 to 7316754839369625600. Current Snapshot: MvccSnapshot[applied={T|T < 7316754839425744896}]
I20260809 23:11:39.478253 30943 mvcc.cc:205] Tried to move back new op lower bound from 7316754839425744896 to 7316754839369625600. Current Snapshot: MvccSnapshot[applied={T|T < 7316754839425744896}]
I20260809 23:11:39.479876 30074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:11:39.479986 30074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:11:39.480885 30074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1597efbef8674bd5929b1a5bceaae78c: Sending DeleteTablet for 3 replicas of tablet 8a6839c4f507474398485b8244f97792
I20260809 23:11:39.481458 30310 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a6839c4f507474398485b8244f97792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:39 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:42326
I20260809 23:11:39.481834 30749 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a6839c4f507474398485b8244f97792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:39 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:54384
I20260809 23:11:39.482419 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 30136
I20260809 23:11:39.489723 30956 tablet_replica.cc:330] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: stopping tablet replica
I20260809 23:11:39.490008 30956 raft_consensus.cc:2229] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:11:39.490206 30956 raft_consensus.cc:2258] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:39.492390 30956 ts_tablet_manager.cc:1905] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:11:39.497102 30956 ts_tablet_manager.cc:1918] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:11:39.497186 30956 log.cc:1196] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/8a6839c4f507474398485b8244f97792
I20260809 23:11:39.497506 30956 ts_tablet_manager.cc:1939] T 8a6839c4f507474398485b8244f97792 P 6d4c1221b6d74d69a9159d486f5ed75f: Deleting consensus metadata
I20260809 23:11:39.500501 30529 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a6839c4f507474398485b8244f97792 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:39 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:60626
I20260809 23:11:39.505481 30957 tablet_replica.cc:330] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: stopping tablet replica
I20260809 23:11:39.505703 30957 raft_consensus.cc:2229] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:11:39.505910 30957 raft_consensus.cc:2258] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:39.507536 30957 ts_tablet_manager.cc:1905] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:11:39.508036 30048 catalog_manager.cc:4841] TS 6d4c1221b6d74d69a9159d486f5ed75f (127.8.141.195:41673): tablet 8a6839c4f507474398485b8244f97792 (table test-table [id=39f0c5e491be46368598a507457784a8]) successfully deleted
I20260809 23:11:39.509614 30957 ts_tablet_manager.cc:1918] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:11:39.509766 30957 log.cc:1196] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/8a6839c4f507474398485b8244f97792
I20260809 23:11:39.510116 30957 ts_tablet_manager.cc:1939] T 8a6839c4f507474398485b8244f97792 P fc04d455f1d84b46b86562f179838cd9: Deleting consensus metadata
W20260809 23:11:39.510349 30047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:40831 (error 108)
I20260809 23:11:39.511384 30050 catalog_manager.cc:4841] TS fc04d455f1d84b46b86562f179838cd9 (127.8.141.194:34485): tablet 8a6839c4f507474398485b8244f97792 (table test-table [id=39f0c5e491be46368598a507457784a8]) successfully deleted
I20260809 23:11:39.511658 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 30425
I20260809 23:11:39.527633 30911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:39.527464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:40831 (local address 127.0.0.1:42334)
0809 23:11:39.527531 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:39.527584 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:40831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 23:11:39.527961 30047 catalog_manager.cc:4584] TS 1cce424fd6c6471ba252d05a0fc8b459 (127.8.141.193:40831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a6839c4f507474398485b8244f97792: Network error: Client connection negotiation failed: client connection to 127.8.141.193:40831: connect: Connection refused (error 111)
I20260809 23:11:39.534225 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 30643
I20260809 23:11:39.554550 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 30023
I20260809 23:11:39.568858 8759 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 23:11:39.928776 8759 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 23:11:39.928887 8759 mini_ranger.cc:66] Stopping Ranger...
I20260809 23:11:40.284677 8759 mini_ranger.cc:68] Stopped Ranger
2026-08-09 23:11:40.284 UTC [28496] LOG: received smart shutdown request
2026-08-09 23:11:40.290 UTC [28496] LOG: background worker "logical replication launcher" (PID 28513) exited with exit code 1
2026-08-09 23:11:40.290 UTC [28508] LOG: shutting down
2026-08-09 23:11:40.504 UTC [28496] LOG: database system is shut down
2026-08-09T23:11:40Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55683 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:11:40 dist-test-slave-j2vl krb5kdc[30976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:11:40 dist-test-slave-j2vl krb5kdc[30976](info): set up 2 sockets
Aug 09 23:11:40 dist-test-slave-j2vl krb5kdc[30976](info): commencing operation
krb5kdc: starting...
W20260809 23:11:41.958837 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:11:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:11:42.096532 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39137
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:39137,127.8.141.253:38715,127.8.141.252:37087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:42.181911 30995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:42.182168 30995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:42.182209 30995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:42.183420 30995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:42.183490 30995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:42.183513 30995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:42.183535 30995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:42.183564 30995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:42.185196 30995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:39137,127.8.141.253:38715,127.8.141.252:37087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:42.185525 30995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:42.185768 30995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:42.190903 31011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:42.190858 31008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:42.190920 30995 server_base.cc:1034] running on GCE node
W20260809 23:11:42.190884 31006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:42.191437 30995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:42.191776 30995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:42.192967 30995 hybrid_clock.cc:648] HybridClock initialized: now 1786317102192947 us; error 46 us; skew 500 ppm
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:42.195665 30995 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:11:42.196173 30995 webserver.cc:460] Webserver started at http://127.8.141.254:38429/ using document root <none> and password file <none>
I20260809 23:11:42.196440 30995 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:42.196516 30995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:42.196681 30995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:42.197966 30995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "eb2fe5481cd843daafa1d20673da9c23"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "eaf30cd5f8c1c1fa69413199458d6734"
server_key_iv: "cb0aa25ff3cf6aed752d245d4fd9e860"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.198374 30995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "eb2fe5481cd843daafa1d20673da9c23"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "eaf30cd5f8c1c1fa69413199458d6734"
server_key_iv: "cb0aa25ff3cf6aed752d245d4fd9e860"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.199892 30995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:11:42.200721 31021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:42.200908 30995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:11:42.201023 30995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "eb2fe5481cd843daafa1d20673da9c23"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "eaf30cd5f8c1c1fa69413199458d6734"
server_key_iv: "cb0aa25ff3cf6aed752d245d4fd9e860"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.201104 30995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:42.210561 30995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:42.212984 30995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:42.213182 30995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:42.217821 30995 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:39137
I20260809 23:11:42.217875 31106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:39137 every 8 connection(s)
I20260809 23:11:42.218284 30995 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:11:42.219283 31108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:42.220949 31108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.222610 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 30995
I20260809 23:11:42.222738 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:11:42.222954 8759 external_mini_cluster.cc:1428] Setting key c0d926ffd2ebebd0436b1bb36fa74d1e
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.141.253@KRBTEST.COM; defaulting to no policy
I20260809 23:11:42.232425 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.221726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39137 (local address 127.0.0.1:45858)
0809 23:11:42.221909 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.221950 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.222445 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.222795 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.222798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.223135 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.223299 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.223303 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.224842 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.224843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.226091 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.226094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.226633 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.226634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.226699 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.227299 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.227308 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.228740 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.230809 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.230816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.230819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.231026 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.231289 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.231290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.231290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.231379 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.231784 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.231790 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.231861 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.231959 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.232209 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":105,"thread_start_us":96,"threads_started":1}
I20260809 23:11:42.232434 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.221767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.8.141.254:39137)
0809 23:11:42.221945 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.221954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.222045 (+ 91us) server_negotiation.cc:407] Connection header received
0809 23:11:42.222610 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.222613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.222637 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.222794 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.223412 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.224718 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.226186 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.226540 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.226638 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.228831 (+ 2193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.228840 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.228840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.228848 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.230637 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.231127 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.231130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.231130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.231217 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.231459 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.231461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.231461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.231695 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.231804 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.231984 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.232160 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":134,"thread_start_us":107,"threads_started":1}
I20260809 23:11:42.233263 31108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.233755 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.233581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57158)
0809 23:11:42.233673 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.233725 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 23:11:42.233983 31027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
I20260809 23:11:42.234476 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.234346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57162)
0809 23:11:42.234424 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.234455 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 23:11:42.234687 31108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:38715: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab.
Entry for principal kudu/127.8.141.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab.
I20260809 23:11:42.258591 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:38715
--webserver_interface=127.8.141.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab
--principal=kudu/127.8.141.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:39137,127.8.141.253:38715,127.8.141.252:37087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:42.283986 31108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } attempt: 1
I20260809 23:11:42.284592 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.284449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57164)
0809 23:11:42.284525 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.284560 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 23:11:42.284932 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.284840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57166)
0809 23:11:42.284883 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.284908 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 23:11:42.285017 31108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:38715: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
W20260809 23:11:42.344897 31117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:42.345162 31117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:42.345201 31117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:42.346449 31117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:42.346514 31117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:42.346541 31117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:42.346571 31117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:42.346597 31117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:42.348176 31117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:39137,127.8.141.253:38715,127.8.141.252:37087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab
--principal=kudu/127.8.141.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:38715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.141.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:42.348462 31117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:42.348694 31117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:11:42.353164 31108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } attempt: 2
I20260809 23:11:42.353749 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.353544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57182)
0809 23:11:42.353638 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.353693 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 23:11:42.354146 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.354048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57186)
0809 23:11:42.354093 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.354118 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 23:11:42.354102 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:42.354161 31134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:42.354240 31108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:38715: Network error: Client connection negotiation failed: client connection to 127.8.141.253:38715: connect: Connection refused (error 111)
W20260809 23:11:42.354406 31132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:42.354405 31117 server_base.cc:1034] running on GCE node
I20260809 23:11:42.354733 31117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:42.355043 31117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:42.356245 31117 hybrid_clock.cc:648] HybridClock initialized: now 1786317102356162 us; error 54 us; skew 500 ppm
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:42.359020 31117 init.cc:377] Logged in from keytab as kudu/127.8.141.253@KRBTEST.COM (short username kudu)
I20260809 23:11:42.359465 31117 webserver.cc:460] Webserver started at http://127.8.141.253:46301/ using document root <none> and password file <none>
I20260809 23:11:42.359685 31117 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:42.359732 31117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:42.359905 31117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:42.360991 31117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data/instance:
uuid: "a1859ec72cd94ccfa401a0e413f667f9"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "7b5355b879e1fa6afd79b14ecfa9ce29"
server_key_iv: "2ac348982b0b9e795b6e6b4cbf64f6ed"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.361382 31117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/wal/instance:
uuid: "a1859ec72cd94ccfa401a0e413f667f9"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "7b5355b879e1fa6afd79b14ecfa9ce29"
server_key_iv: "2ac348982b0b9e795b6e6b4cbf64f6ed"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.362726 31117 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:11:42.363430 31146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:42.363593 31117 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:42.363682 31117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/wal
uuid: "a1859ec72cd94ccfa401a0e413f667f9"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "7b5355b879e1fa6afd79b14ecfa9ce29"
server_key_iv: "2ac348982b0b9e795b6e6b4cbf64f6ed"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.363735 31117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:42.381767 31117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:42.384150 31117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:42.384279 31117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:42.388659 31117 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.253:38715
I20260809 23:11:42.388721 31235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.253:38715 every 8 connection(s)
I20260809 23:11:42.389060 31117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
I20260809 23:11:42.389987 31236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:42.391566 31236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.395131 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 31117
I20260809 23:11:42.395246 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-1/wal/instance
I20260809 23:11:42.395408 8759 external_mini_cluster.cc:1428] Setting key 51797f9253cbd040d7539b64e583e403
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:42.402034 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.392298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39137 (local address 127.0.0.1:45870)
0809 23:11:42.392480 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.392516 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.392989 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.393250 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.393254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.393533 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.393728 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.393733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.394447 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.394449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.395522 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.395526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.395929 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.395930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.396086 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.396633 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.396643 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.398595 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.400595 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.400603 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.400605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.400825 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.401053 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.401054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.401054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.401152 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.401514 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.401520 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.401639 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.401721 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.401845 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":85,"threads_started":1}
I20260809 23:11:42.402037 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.392312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45870 (local address 127.8.141.254:39137)
0809 23:11:42.392384 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.392386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.392603 (+ 217us) server_negotiation.cc:407] Connection header received
0809 23:11:42.393096 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.393098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.393114 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.393179 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.393828 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.394332 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.395605 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.395837 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.395905 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.398696 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.398698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.398699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.398709 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.400509 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.400905 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.400906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.400907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.400982 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.401228 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.401229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.401230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.401432 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.401515 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.401776 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.401885 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:11:42.402771 31236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.141.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.252@KRBTEST.COM" created.
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:11:42.412582 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.403077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57200)
0809 23:11:42.403129 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.403141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.403187 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.403459 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.403459 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.403617 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.403685 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.403687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.404674 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.404675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.405185 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.405187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.405747 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.405748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.405816 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.406234 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.406239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.409683 (+ 3444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.411532 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.411533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.411533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.411759 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.411984 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.411984 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.411985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.412038 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.412353 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.412354 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.412389 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.412441 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.412516 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 23:11:42.412767 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.403160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.8.141.253:38715)
0809 23:11:42.403322 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.403326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.403348 (+ 22us) server_negotiation.cc:407] Connection header received
0809 23:11:42.403365 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.403366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.403376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.403458 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.403755 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.404579 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.405264 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.405674 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.405751 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.409787 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.409796 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.409796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.409803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.411453 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.411846 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.411850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.411850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.411915 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.412098 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.412099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.412099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.412290 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.412398 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.412478 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.412607 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":132,"thread_start_us":105,"threads_started":1}
I20260809 23:11:42.413560 31236 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.413937 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.413807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:41990)
0809 23:11:42.413874 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.413919 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 23:11:42.414074 31147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
I20260809 23:11:42.414529 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.414438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42000)
0809 23:11:42.414484 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.414508 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:11:42.414633 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37087: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab.
Entry for principal kudu/127.8.141.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab.
I20260809 23:11:42.428076 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:37087
--webserver_interface=127.8.141.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab
--principal=kudu/127.8.141.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:39137,127.8.141.253:38715,127.8.141.252:37087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:42.445431 31108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } attempt: 3
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:11:42.453047 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.445929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57208 (local address 127.8.141.253:38715)
0809 23:11:42.446000 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.446003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.446017 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:11:42.446095 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.446096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.446115 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.446181 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.446555 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.447139 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.447912 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.448141 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.448223 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.450288 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.450290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.450290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.450299 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.451806 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.452201 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.452203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.452203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.452268 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.452475 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.452477 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.452477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.452600 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.452678 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.452850 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.452940 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 23:11:42.453047 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.445849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57208)
0809 23:11:42.445923 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.445942 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.446017 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.446188 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.446189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.446385 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.446457 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.446460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.447246 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.447247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.447822 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.447823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.448220 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.448221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.448315 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.448813 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.448818 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.450187 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.451931 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.451932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.451932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.452126 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.452337 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.452338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.452338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.452409 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.452679 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.452681 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.452734 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.452822 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.452936 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 23:11:42.453516 31108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.454010 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.453864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42008)
0809 23:11:42.453948 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.453984 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 23:11:42.454381 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.454285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42020)
0809 23:11:42.454328 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.454353 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 23:11:42.454476 31108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37087: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
I20260809 23:11:42.463858 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } attempt: 1
I20260809 23:11:42.465991 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.464198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42026)
0809 23:11:42.465918 (+ 1720us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.465959 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1696}
I20260809 23:11:42.467478 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.466358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42030)
0809 23:11:42.467411 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.467447 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1032}
W20260809 23:11:42.467581 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37087: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
I20260809 23:11:42.485621 31108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } attempt: 1
I20260809 23:11:42.486114 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.485982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42044)
0809 23:11:42.486048 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.486086 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 23:11:42.486443 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.486352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42054)
0809 23:11:42.486396 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.486421 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 23:11:42.486528 31108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37087: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
W20260809 23:11:42.514662 31248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:42.514914 31248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:42.514950 31248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:42.516055 31248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:42.516119 31248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:42.516145 31248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:42.516175 31248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:42.516204 31248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:42.517827 31248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:39137,127.8.141.253:38715,127.8.141.252:37087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab
--principal=kudu/127.8.141.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:37087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.141.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:42.518106 31248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:42.518350 31248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:42.523324 31256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:42.523339 31259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:42.523504 31257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:42.523525 31248 server_base.cc:1034] running on GCE node
I20260809 23:11:42.523921 31248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:42.524206 31248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:42.525453 31248 hybrid_clock.cc:648] HybridClock initialized: now 1786317102525421 us; error 34 us; skew 500 ppm
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:42.528203 31248 init.cc:377] Logged in from keytab as kudu/127.8.141.252@KRBTEST.COM (short username kudu)
I20260809 23:11:42.528743 31248 webserver.cc:460] Webserver started at http://127.8.141.252:40669/ using document root <none> and password file <none>
I20260809 23:11:42.529037 31248 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:42.529115 31248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:42.529269 31248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:42.530452 31248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data/instance:
uuid: "071316f82dda49a7b9a2277c4af3660d"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "440958225b8913be5136fc02fc33ad7b"
server_key_iv: "215b7ac5ce278fe31361f33a6bc9bdff"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.530830 31248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/wal/instance:
uuid: "071316f82dda49a7b9a2277c4af3660d"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "440958225b8913be5136fc02fc33ad7b"
server_key_iv: "215b7ac5ce278fe31361f33a6bc9bdff"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.532347 31248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:11:42.533249 31270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:42.533475 31248 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:42.533565 31248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/wal
uuid: "071316f82dda49a7b9a2277c4af3660d"
format_stamp: "Formatted at 2026-08-09 23:11:42 on dist-test-slave-j2vl"
server_key: "440958225b8913be5136fc02fc33ad7b"
server_key_iv: "215b7ac5ce278fe31361f33a6bc9bdff"
server_key_version: "encryptionkey@0"
I20260809 23:11:42.533614 31248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:42.535715 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } attempt: 2
I20260809 23:11:42.536142 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.536016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42070)
0809 23:11:42.536075 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.536112 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 23:11:42.536597 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.536463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42080)
0809 23:11:42.536529 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.536572 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 23:11:42.536724 31236 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37087: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37087: connect: Connection refused (error 111)
I20260809 23:11:42.551213 31248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:42.553151 31248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:42.553313 31248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:42.557431 31248 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.252:37087
I20260809 23:11:42.557483 31368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.252:37087 every 8 connection(s)
I20260809 23:11:42.557905 31248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
I20260809 23:11:42.558779 31369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:42.560330 31369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.561668 31108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } attempt: 2
I20260809 23:11:42.563977 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 31248
I20260809 23:11:42.564080 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786316926160006-8759-0/minicluster-data/master-2/wal/instance
I20260809 23:11:42.564268 8759 external_mini_cluster.cc:1428] Setting key 6e23720871a339947b1cd628d6198751
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
I20260809 23:11:42.570801 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.561067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.8.141.254:39137)
0809 23:11:42.561134 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.561136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.561379 (+ 243us) server_negotiation.cc:407] Connection header received
0809 23:11:42.561928 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.561930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.561941 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.561980 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.562854 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.563346 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.564528 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.564881 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.564959 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.567567 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.567569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.567569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.567578 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.569301 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.569751 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.569752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.569753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.569811 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.570049 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.570051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.570051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.570203 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.570292 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.570619 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.570696 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:11:42.570977 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.561024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39137 (local address 127.0.0.1:45884)
0809 23:11:42.561212 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.561255 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.561816 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.562165 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.562169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.562540 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.562767 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.562772 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.563426 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.563427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.564422 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.564425 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.565015 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.565016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.565160 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.565908 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.565919 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.567460 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.569435 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.569443 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.569445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.569662 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.569886 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.569886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.569887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.569986 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.570322 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.570330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.570491 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.570599 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.570745 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260809 23:11:42.571602 31369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.571837 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.562010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42086)
0809 23:11:42.562045 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.562055 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.562084 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.562595 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.562596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.562761 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.562807 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.562809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.563919 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.563921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.564721 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.564724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.565466 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.565468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.565544 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.566008 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.566014 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.567924 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.570247 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.570248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.570248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.570448 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.570703 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.570704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.570705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.570794 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.571550 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.571551 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.571589 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.571628 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.571724 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 23:11:42.572492 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.562215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.8.141.252:37087)
0809 23:11:42.562417 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.562421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.562441 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:11:42.562462 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.562464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.562486 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.562567 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.562929 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.563828 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.564807 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.565371 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.565447 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.568024 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.568033 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.568034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.568042 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.570165 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.570531 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.570534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.570535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.570603 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.571280 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.571281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.571281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.571478 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.572155 (+ 677us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.572206 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.572346 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":168,"thread_start_us":114,"threads_started":1}
I20260809 23:11:42.574807 31108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: Bootstrap starting.
I20260809 23:11:42.574939 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.564644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42100 (local address 127.8.141.252:37087)
0809 23:11:42.564772 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.564773 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.564891 (+ 118us) server_negotiation.cc:407] Connection header received
0809 23:11:42.564959 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.564960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.564971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.565061 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.565936 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.566709 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.567612 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.567871 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.567938 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.570979 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.570981 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.570981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.570988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.573653 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.574046 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.574047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.574047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.574104 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.574367 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.574369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.574369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.574482 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.574594 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.574732 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.574796 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260809 23:11:42.575626 31108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:42.575977 31108 log.cc:824] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:42.576881 31108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: No bootstrap required, opened a new log
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:11:42.579507 31108 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.579664 31108 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:42.579700 31108 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2fe5481cd843daafa1d20673da9c23, State: Initialized, Role: FOLLOWER
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:42.579828 31108 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.580137 31379 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.580197 31108 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:42.580235 31379 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:11:42.583165 31399 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:11:42.583223 31399 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:11:42.583423 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.571896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:38715 (local address 127.0.0.1:57218)
0809 23:11:42.571996 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.572007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.572048 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.572232 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.572233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.572433 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.572497 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.572499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.576277 (+ 3778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.576278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.577049 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.577051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.577596 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.577598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.577671 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.578109 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.578115 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.580155 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.582322 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.582323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.582324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.582489 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.582736 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.582737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.582738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.582801 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.583106 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.583107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.583152 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.583214 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.583327 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 23:11:42.583709 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.572001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.8.141.253:38715)
0809 23:11:42.572057 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.572060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.572071 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:11:42.572100 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.572101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.572112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.572170 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.572586 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.573217 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.577161 (+ 3944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.577465 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.577515 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.580273 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.580275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.580275 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.580284 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.582223 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.582591 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.582592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.582593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.582659 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.582873 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.582875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.582875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.583013 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.583078 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.583545 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.583628 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:11:42.583971 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.575637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45888 (local address 127.8.141.254:39137)
0809 23:11:42.575673 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.575676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.575690 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:11:42.575763 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.575764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.575773 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.575829 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.576420 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.576946 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.577706 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.577903 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.577975 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.580663 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.580665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.580665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.580672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.582381 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.582771 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.582772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.582773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.582830 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.583040 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.583041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.583041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.583166 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.583288 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.583750 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.583832 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:11:42.584040 31369 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:11:42.583896 30995 master_runner.cc:186] Error getting master registration for 127.8.141.253:38715: OK, instance_id { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" instance_seqno: 1786317102387460 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:11:42.586876 30995 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:11:42.591504 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.584305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42108)
0809 23:11:42.584357 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.584369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.584410 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.584547 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.584548 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.584697 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.584732 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.584734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.585728 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.585729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.586446 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.586448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.586929 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.586930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.586977 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.587438 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.587442 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.588578 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.590398 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.590398 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.590399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.590595 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.590793 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.590794 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.590794 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.590863 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.591208 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.591209 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.591246 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.591304 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.591407 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:11:42.591895 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.584387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42108 (local address 127.8.141.252:37087)
0809 23:11:42.584434 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.584436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.584444 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:11:42.584462 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.584463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.584473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.584536 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.584811 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.585626 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.586531 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.586818 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.586881 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.588659 (+ 1778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.588660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.588660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.588667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.590296 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.590658 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.590659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.590660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.590720 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.590938 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.590939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.590940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.591123 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.591186 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.591736 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.591816 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 23:11:42.592657 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.584836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57234 (local address 127.8.141.253:38715)
0809 23:11:42.584879 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.584881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.584898 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:11:42.584918 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.584918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.584927 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.584971 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.585455 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.586103 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.587030 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.587312 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.587381 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.589917 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.589919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.589919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.589926 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.591281 (+ 1355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.591647 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.591648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.591649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.591705 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.591882 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.591883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.591883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.592021 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.592274 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.592476 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.592564 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:11:42.593565 31369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d: Bootstrap starting.
I20260809 23:11:42.594383 31369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:42.594762 31369 log.cc:824] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:42.594938 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:11:42.595604 31369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d: No bootstrap required, opened a new log
I20260809 23:11:42.598040 31369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.598208 31369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:42.598258 31369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071316f82dda49a7b9a2277c4af3660d, State: Initialized, Role: FOLLOWER
I20260809 23:11:42.598400 31369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.598711 31369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:42.605122 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.596505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45890 (local address 127.8.141.254:39137)
0809 23:11:42.596555 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.596557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.597524 (+ 967us) server_negotiation.cc:407] Connection header received
0809 23:11:42.597597 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.597598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.597611 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.597670 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.598383 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.599109 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.600088 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.600382 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.600419 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.602098 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.602100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.602100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.602107 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.603812 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.604250 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.604252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.604253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.604313 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.604531 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.604532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.604532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.604663 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.604736 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.604947 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.605025 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:11:42.601924 31248 master_runner.cc:186] Error getting master registration for 127.8.141.253:38715: OK, instance_id { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" instance_seqno: 1786317102387460 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:11:42.605320 31248 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:11:42.600315 31417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.605476 31417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:11:42.605544 31417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:42.605619 31417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.605793 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.596618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42122 (local address 127.8.141.252:37087)
0809 23:11:42.596652 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.596655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.597117 (+ 462us) server_negotiation.cc:407] Connection header received
0809 23:11:42.597166 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.597167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.597178 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.597238 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.598012 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.598681 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.599765 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.600076 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.600123 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.602633 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.602635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.602635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.602644 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.604513 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.604939 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.604940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.604941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.604996 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.605186 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.605187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.605187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.605303 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.605402 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.605628 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.605707 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:11:42.606047 31061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "071316f82dda49a7b9a2277c4af3660d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb2fe5481cd843daafa1d20673da9c23" is_pre_election: true
I20260809 23:11:42.606190 31061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 071316f82dda49a7b9a2277c4af3660d in term 0.
I20260809 23:11:42.606312 31417 leader_election.cc:288] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb2fe5481cd843daafa1d20673da9c23 (127.8.141.254:39137), a1859ec72cd94ccfa401a0e413f667f9 (127.8.141.253:38715)
I20260809 23:11:42.606420 31277 leader_election.cc:302] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 071316f82dda49a7b9a2277c4af3660d, eb2fe5481cd843daafa1d20673da9c23; no voters:
I20260809 23:11:42.606743 31417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:11:42.606882 31417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:11:42.606943 31417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 0 FOLLOWER]: Advancing to term 1
W20260809 23:11:42.607025 31430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:11:42.607070 31430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:11:42.608026 31417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.608304 31061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "071316f82dda49a7b9a2277c4af3660d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb2fe5481cd843daafa1d20673da9c23"
I20260809 23:11:42.608389 31061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:42.608415 31417 leader_election.cc:288] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [CANDIDATE]: Term 1 election: Requested vote from peers eb2fe5481cd843daafa1d20673da9c23 (127.8.141.254:39137), a1859ec72cd94ccfa401a0e413f667f9 (127.8.141.253:38715)
I20260809 23:11:42.609066 31061 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 071316f82dda49a7b9a2277c4af3660d in term 1.
I20260809 23:11:42.609264 31277 leader_election.cc:302] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 071316f82dda49a7b9a2277c4af3660d, eb2fe5481cd843daafa1d20673da9c23; no voters:
I20260809 23:11:42.609405 31417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:11:42.609477 31417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [term 1 LEADER]: Becoming Leader. State: Replica: 071316f82dda49a7b9a2277c4af3660d, State: Running, Role: LEADER
I20260809 23:11:42.609571 31417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.609999 31417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071316f82dda49a7b9a2277c4af3660d. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.610093 31417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:42.611140 31243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.596829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.8.141.253:38715)
0809 23:11:42.596877 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.596879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.597690 (+ 811us) server_negotiation.cc:407] Connection header received
0809 23:11:42.597781 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.597783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.597793 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.597822 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.598709 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.599387 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.600322 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.600608 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.600650 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.607403 (+ 6753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.607404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.607405 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.607412 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.609903 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.610316 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.610317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.610318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.610395 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.610608 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.610608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.610609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.610732 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.610820 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.610953 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.611043 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:11:42.613493 31435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:11:42.613710 31435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:11:42.615274 31061 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [term 1 FOLLOWER]: Refusing update from remote peer 071316f82dda49a7b9a2277c4af3660d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:42.615623 31411 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 23:11:42.617466 31379 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071316f82dda49a7b9a2277c4af3660d. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.617569 31379 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:11:42.627826 31236 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } attempt: 3
Aug 09 23:11:42 dist-test-slave-j2vl krb5kdc[30976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317102, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
I20260809 23:11:42.636605 31375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.628327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42136 (local address 127.8.141.252:37087)
0809 23:11:42.628387 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:42.628390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:42.628398 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:11:42.628422 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:42.628423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:42.628434 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:42.628493 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:42.628840 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.629611 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.630612 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.630940 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.631017 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.633640 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:42.633642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:42.633642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:42.633650 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:42.635289 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.635742 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.635743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.635743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.635826 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:42.636048 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:42.636048 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:42.636049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:42.636202 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:42.636267 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:42.636426 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 23:11:42.636506 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:11:42.636653 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:42.628122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37087 (local address 127.0.0.1:42136)
0809 23:11:42.628182 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:42.628196 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:42.628244 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:42.628505 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:42.628506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:42.628704 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:42.628758 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.628760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.629687 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.629688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.630514 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:42.630517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:42.631029 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:42.631030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:42.631085 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:42.631925 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:42.631931 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:42.633550 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:42.635379 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.635380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.635380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.635652 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.635889 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:42.635889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:42.635890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:42.635980 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:42.636269 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:42.636270 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:42.636319 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 23:11:42.636390 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 23:11:42.636550 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:11:42.638728 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: Bootstrap starting.
I20260809 23:11:42.639531 31236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:42.639846 31236 log.cc:824] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:42.641396 31236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: No bootstrap required, opened a new log
I20260809 23:11:42.642489 31417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.642588 31417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:42.642808 31379 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.642910 31379 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:11:42.644618 31236 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.644770 31236 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:42.644810 31236 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1859ec72cd94ccfa401a0e413f667f9, State: Initialized, Role: FOLLOWER
I20260809 23:11:42.644970 31440 mvcc.cc:205] Tried to move back new op lower bound from 7316754852309692416 to 7316754852289523712. Current Snapshot: MvccSnapshot[applied={T|T < 7316754852309692416}]
I20260809 23:11:42.645097 31236 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } }
I20260809 23:11:42.645507 31236 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:42.645557 31191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "071316f82dda49a7b9a2277c4af3660d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1859ec72cd94ccfa401a0e413f667f9"
I20260809 23:11:42.645677 31191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:42.646595 31191 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 071316f82dda49a7b9a2277c4af3660d in term 1.
I20260809 23:11:42.646821 31192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "071316f82dda49a7b9a2277c4af3660d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1859ec72cd94ccfa401a0e413f667f9" is_pre_election: true
I20260809 23:11:42.646922 31192 raft_consensus.cc:2362] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 071316f82dda49a7b9a2277c4af3660d in term 1. Re-sending same reply.
I20260809 23:11:42.647298 31190 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [term 1 FOLLOWER]: Refusing update from remote peer 071316f82dda49a7b9a2277c4af3660d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:42.647768 31411 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:11:42.650507 31435 catalog_manager.cc:1347] Generated new cluster ID: 7024fdd856314ed88aeb7636557eabf2
I20260809 23:11:42.650573 31435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:11:42.653475 31449 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.653594 31449 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:11:42.653652 31449 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071316f82dda49a7b9a2277c4af3660d. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.653688 31449 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:11:42.653710 31449 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.653744 31449 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:11:42.654769 31467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:11:42.654827 31467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:11:42.668000 31472 mvcc.cc:205] Tried to move back new op lower bound from 7316754852309692416 to 7316754852289523712. Current Snapshot: MvccSnapshot[applied={T|T < 7316754852309692416}]
I20260809 23:11:42.669610 31411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "071316f82dda49a7b9a2277c4af3660d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2fe5481cd843daafa1d20673da9c23" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39137 } } peers { permanent_uuid: "a1859ec72cd94ccfa401a0e413f667f9" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 38715 } } peers { permanent_uuid: "071316f82dda49a7b9a2277c4af3660d" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37087 } } }
I20260809 23:11:42.669721 31411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:42.670749 31435 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:11:42.671511 31435 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:11:42.683188 31435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 071316f82dda49a7b9a2277c4af3660d: Generated new TSK 0
I20260809 23:11:42.683394 31435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:11:43.583760 31399 catalog_manager.cc:1259] Loaded cluster ID: 7024fdd856314ed88aeb7636557eabf2
I20260809 23:11:43.583856 31399 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: loading cluster ID for follower catalog manager: success
I20260809 23:11:43.584648 31399 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: acquiring CA information for follower catalog manager: success
I20260809 23:11:43.584918 31399 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb2fe5481cd843daafa1d20673da9c23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:11:43.655365 31467 catalog_manager.cc:1259] Loaded cluster ID: 7024fdd856314ed88aeb7636557eabf2
I20260809 23:11:43.655463 31467 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: loading cluster ID for follower catalog manager: success
I20260809 23:11:43.656579 31467 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: acquiring CA information for follower catalog manager: success
I20260809 23:11:43.657042 31467 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a1859ec72cd94ccfa401a0e413f667f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:11:43.743058 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 30995
I20260809 23:11:43.758939 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 31117
I20260809 23:11:43.772342 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 31248
2026-08-09T23:11:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3136 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:11:43 dist-test-slave-j2vl krb5kdc[31494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:11:43 dist-test-slave-j2vl krb5kdc[31494](info): set up 2 sockets
Aug 09 23:11:43 dist-test-slave-j2vl krb5kdc[31494](info): commencing operation
krb5kdc: starting...
W20260809 23:11:45.096436 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:11:45 dist-test-slave-j2vl krb5kdc[31494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:11:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:11:45.236080 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44191
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:44191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:45.322878 31518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:45.323098 31518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:45.323122 31518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:45.324234 31518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:45.324267 31518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:45.324281 31518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:45.324293 31518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:45.324306 31518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:45.325800 31518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:44191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:45.326006 31518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:45.326182 31518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:45.331162 31528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:45.331194 31524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:45.331233 31525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:45.331538 31518 server_base.cc:1034] running on GCE node
I20260809 23:11:45.331801 31518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:45.332113 31518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:45.333305 31518 hybrid_clock.cc:648] HybridClock initialized: now 1786317105333298 us; error 52 us; skew 500 ppm
Aug 09 23:11:45 dist-test-slave-j2vl krb5kdc[31494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317105, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:45.336092 31518 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:11:45.336624 31518 webserver.cc:460] Webserver started at http://127.8.141.254:34765/ using document root <none> and password file <none>
I20260809 23:11:45.336870 31518 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:45.336943 31518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:45.337095 31518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:45.338291 31518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "d0fbd7f92f11407a9e115e70da3e6ebb"
format_stamp: "Formatted at 2026-08-09 23:11:45 on dist-test-slave-j2vl"
server_key: "514694cfa4265f28baa4f60790f28ec5"
server_key_iv: "25fd21912e4c87df8e7fbe4036c9bb0c"
server_key_version: "encryptionkey@0"
I20260809 23:11:45.338686 31518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "d0fbd7f92f11407a9e115e70da3e6ebb"
format_stamp: "Formatted at 2026-08-09 23:11:45 on dist-test-slave-j2vl"
server_key: "514694cfa4265f28baa4f60790f28ec5"
server_key_iv: "25fd21912e4c87df8e7fbe4036c9bb0c"
server_key_version: "encryptionkey@0"
I20260809 23:11:45.340124 31518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:11:45.341004 31539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:45.341183 31518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:45.341288 31518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "d0fbd7f92f11407a9e115e70da3e6ebb"
format_stamp: "Formatted at 2026-08-09 23:11:45 on dist-test-slave-j2vl"
server_key: "514694cfa4265f28baa4f60790f28ec5"
server_key_iv: "25fd21912e4c87df8e7fbe4036c9bb0c"
server_key_version: "encryptionkey@0"
I20260809 23:11:45.341415 31518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:45.358156 31518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:45.360513 31518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:45.360713 31518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:45.365913 31518 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:44191
I20260809 23:11:45.365962 31621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:44191 every 8 connection(s)
I20260809 23:11:45.366395 31518 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:11:45.367388 31622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:45.369822 31622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb: Bootstrap starting.
I20260809 23:11:45.370484 31622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:45.370793 31622 log.cc:824] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:45.371598 31622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb: No bootstrap required, opened a new log
I20260809 23:11:45.372449 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 31518
I20260809 23:11:45.372560 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:11:45.372749 8759 external_mini_cluster.cc:1428] Setting key 7b6cbee58e0c7502908edc2dbad8a4ef
I20260809 23:11:45.373943 31622 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44191 } }
I20260809 23:11:45.374119 31622 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:45.374161 31622 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0fbd7f92f11407a9e115e70da3e6ebb, State: Initialized, Role: FOLLOWER
I20260809 23:11:45.374315 31622 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44191 } }
I20260809 23:11:45.374413 31622 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:11:45.374455 31622 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:11:45.374509 31622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:45.375582 31622 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44191 } }
I20260809 23:11:45.375682 31622 leader_election.cc:302] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0fbd7f92f11407a9e115e70da3e6ebb; no voters:
I20260809 23:11:45.375872 31622 leader_election.cc:288] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:11:45.375921 31630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:11:45.376078 31630 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [term 1 LEADER]: Becoming Leader. State: Replica: d0fbd7f92f11407a9e115e70da3e6ebb, State: Running, Role: LEADER
I20260809 23:11:45.376160 31622 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:45.376194 31630 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44191 } }
I20260809 23:11:45.376544 31631 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44191 } } }
I20260809 23:11:45.376641 31631 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:45.376857 31639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:11:45.376902 31632 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0fbd7f92f11407a9e115e70da3e6ebb. Latest consensus state: current_term: 1 leader_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0fbd7f92f11407a9e115e70da3e6ebb" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44191 } } }
I20260809 23:11:45.377069 31632 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:45.377411 31639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:11:45 dist-test-slave-j2vl krb5kdc[31494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:45.380559 31639 catalog_manager.cc:1347] Generated new cluster ID: cc5c142092484a9d923bc8c62d406d79
I20260809 23:11:45.380607 31639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:11:45.385053 31629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:45.373322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36614 (local address 127.8.141.254:44191)
0809 23:11:45.373604 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:45.373612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:45.373646 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:11:45.373917 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:45.373923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:45.373972 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:45.374084 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:45.374932 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:45.375997 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:45.377111 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:45.377726 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:45.377844 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:45.380981 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:45.380986 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:45.380986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:45.380999 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:45.383106 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:45.383507 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:45.383510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:45.383510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:45.383659 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:45.383871 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:45.383872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:45.383873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:45.384209 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:45.384394 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:45.384468 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 23:11:45.384617 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":129,"threads_started":1}
I20260809 23:11:45.386161 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:11:45.388002 31639 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:11:45.388756 31639 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:11:45.393486 31639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0fbd7f92f11407a9e115e70da3e6ebb: Generated new TSK 0
I20260809 23:11:45.393725 31639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:11:45.440759 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 31518
2026-08-09T23:11:45Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1670 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:11:45 dist-test-slave-j2vl krb5kdc[31666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:11:45 dist-test-slave-j2vl krb5kdc[31666](info): set up 2 sockets
Aug 09 23:11:45 dist-test-slave-j2vl krb5kdc[31666](info): commencing operation
krb5kdc: starting...
W20260809 23:11:46.760497 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.293s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:11:46 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:11:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:11:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:11:46.905257 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41297
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:46.997480 31684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:46.997756 31684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:46.997800 31684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:46.999029 31684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:46.999101 31684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:46.999125 31684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:46.999147 31684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:46.999176 31684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:47.000774 31684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:41297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:47.001077 31684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:47.001325 31684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:47.007146 31698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:47.007212 31684 server_base.cc:1034] running on GCE node
W20260809 23:11:47.007216 31696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.007146 31695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:47.007771 31684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:47.008131 31684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:47.009312 31684 hybrid_clock.cc:648] HybridClock initialized: now 1786317107009299 us; error 44 us; skew 500 ppm
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:47.012195 31684 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:11:47.012758 31684 webserver.cc:460] Webserver started at http://127.8.141.254:33757/ using document root <none> and password file <none>
I20260809 23:11:47.013019 31684 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:47.013087 31684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:47.013235 31684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:47.014395 31684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "8e9ba9c1aa134e268f37bd8ae09882db"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "736f728f03a4ee0bac922d8927d56523"
server_key_iv: "9030aa7e5cfe67407c79ba2548d6ded1"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.014778 31684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "8e9ba9c1aa134e268f37bd8ae09882db"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "736f728f03a4ee0bac922d8927d56523"
server_key_iv: "9030aa7e5cfe67407c79ba2548d6ded1"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.016325 31684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:11:47.017151 31706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.017329 31684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:47.017477 31684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "8e9ba9c1aa134e268f37bd8ae09882db"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "736f728f03a4ee0bac922d8927d56523"
server_key_iv: "9030aa7e5cfe67407c79ba2548d6ded1"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.017557 31684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:47.029332 31684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:47.031988 31684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:47.032181 31684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:47.036645 31684 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:41297
I20260809 23:11:47.036702 31794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:41297 every 8 connection(s)
I20260809 23:11:47.037086 31684 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:11:47.038178 31796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:47.040453 31796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Bootstrap starting.
I20260809 23:11:47.041033 31796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:47.041276 31796 log.cc:824] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:47.041996 31796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: No bootstrap required, opened a new log
I20260809 23:11:47.042250 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 31684
I20260809 23:11:47.042366 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:11:47.042569 8759 external_mini_cluster.cc:1428] Setting key 594558a5298ec42186b807a30dff4f09
I20260809 23:11:47.044358 31796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.044543 31796 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:47.044584 31796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ba9c1aa134e268f37bd8ae09882db, State: Initialized, Role: FOLLOWER
I20260809 23:11:47.044718 31796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.044811 31796 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:11:47.044835 31796 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:11:47.044905 31796 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:47.046083 31796 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.046200 31796 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9ba9c1aa134e268f37bd8ae09882db; no voters:
I20260809 23:11:47.046353 31796 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:11:47.046439 31805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:11:47.046605 31796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:47.046725 31805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ba9c1aa134e268f37bd8ae09882db, State: Running, Role: LEADER
I20260809 23:11:47.046831 31805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.047178 31805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9ba9c1aa134e268f37bd8ae09882db. Latest consensus state: current_term: 1 leader_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } } }
I20260809 23:11:47.047258 31805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:47.047204 31809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } } }
I20260809 23:11:47.047359 31809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: This master's current role is: LEADER
W20260809 23:11:47.048555 31829 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:11:47.048616 31829 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:11:47.048699 31821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:11:47.048969 31821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:47.053866 31821 catalog_manager.cc:1347] Generated new cluster ID: 71ee956d368a4d07938acfe561f22b68
I20260809 23:11:47.053931 31821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:11:47.055303 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.043132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.8.141.254:41297)
0809 23:11:47.043383 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.043393 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.043431 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:11:47.044311 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.044320 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.044387 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.044543 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:47.045383 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.046822 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.048043 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.048677 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.048804 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.051493 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:47.051497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:47.051498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:47.051516 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:47.053309 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.053887 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.053891 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.053891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.054030 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.054323 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.054324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.054325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.054646 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:47.054810 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.054955 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.055100 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":107,"threads_started":1}
I20260809 23:11:47.069506 31821 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:11:47.070147 31821 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:11:47.077971 31821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Generated new TSK 0
I20260809 23:11:47.078218 31821 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:11:47.121558 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41297
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:47.214154 31836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:47.214395 31836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:47.214422 31836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:47.215601 31836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:47.215642 31836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:47.215669 31836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:11:47.217218 31836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:47.217507 31836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:47.217720 31836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:11:47.223281 31836 server_base.cc:1034] running on GCE node
W20260809 23:11:47.223243 31846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.223263 31842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.223263 31848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:47.223721 31836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:47.224018 31836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:47.225200 31836 hybrid_clock.cc:648] HybridClock initialized: now 1786317107225195 us; error 36 us; skew 500 ppm
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:47.228169 31836 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:11:47.228798 31836 webserver.cc:460] Webserver started at http://127.8.141.193:37997/ using document root <none> and password file <none>
I20260809 23:11:47.229064 31836 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:47.229141 31836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:47.229316 31836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:47.230574 31836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "852711c331424839a7bbe8efc0d2ed0b"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "9aca2db98d1df1c6d358054fef081bf7"
server_key_iv: "7f06002815eb7ea95f23ad34fe8e3fc6"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.230991 31836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "852711c331424839a7bbe8efc0d2ed0b"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "9aca2db98d1df1c6d358054fef081bf7"
server_key_iv: "7f06002815eb7ea95f23ad34fe8e3fc6"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.232667 31836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:11:47.233599 31859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.233822 31836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:47.233948 31836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "852711c331424839a7bbe8efc0d2ed0b"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "9aca2db98d1df1c6d358054fef081bf7"
server_key_iv: "7f06002815eb7ea95f23ad34fe8e3fc6"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.234048 31836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:47.279139 31836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:47.281648 31836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:47.281836 31836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:47.282124 31836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:47.282460 31836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:47.282513 31836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.282557 31836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:47.282590 31836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.289183 31836 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:45683
I20260809 23:11:47.289238 32043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:45683 every 8 connection(s)
I20260809 23:11:47.289641 31836 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:47.298908 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 31836
I20260809 23:11:47.299052 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:11:47.299273 8759 external_mini_cluster.cc:1428] Setting key b0e00793a737dbecf9722f65c52231dd
I20260809 23:11:47.299964 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.290637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:44663 (local address 127.8.141.254:41297)
0809 23:11:47.290716 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.290719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.290939 (+ 220us) server_negotiation.cc:407] Connection header received
0809 23:11:47.291500 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.291502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.291514 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.291584 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:47.292652 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.293175 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.294319 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.294560 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.294627 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.296937 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:47.296938 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:47.296939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:47.296949 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:47.298284 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.298785 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.298786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.298787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.298845 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.299176 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.299177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.299177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.299355 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:47.299451 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.299761 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.299865 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:11:47.300185 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.290606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:44663)
0809 23:11:47.290809 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.290851 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.291371 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.291645 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.291648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.292320 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:47.292543 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.292549 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.293287 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.293292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.294198 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.294201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.294648 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.294649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.294770 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.295315 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:47.295326 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:47.296841 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:47.298388 (+ 1547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:47.298396 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:47.298398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:47.298692 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:47.298925 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:47.298927 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:47.298927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:47.299103 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:47.299453 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:47.299462 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:47.299612 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.299721 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.299928 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260809 23:11:47.300722 32044 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41297
I20260809 23:11:47.300823 32044 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:47.301033 32044 heartbeater.cc:511] Master 127.8.141.254:41297 requested a full tablet report, sending...
I20260809 23:11:47.301640 31736 ts_manager.cc:194] Registered new tserver with Master: 852711c331424839a7bbe8efc0d2ed0b (127.8.141.193:45683)
I20260809 23:11:47.302490 31736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:44663
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:11:47.328946 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41297
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:47.418504 32051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:47.418781 32051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:47.418835 32051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:47.420204 32051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:47.420300 32051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:47.420477 32051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:11:47.422335 32051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:47.422640 32051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:47.422904 32051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:11:47.428386 32051 server_base.cc:1034] running on GCE node
W20260809 23:11:47.428457 32066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.428356 32060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.428376 32058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:47.428882 32051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:47.429144 32051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:47.430318 32051 hybrid_clock.cc:648] HybridClock initialized: now 1786317107430304 us; error 35 us; skew 500 ppm
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:47.432946 32051 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:11:47.433516 32051 webserver.cc:460] Webserver started at http://127.8.141.194:39419/ using document root <none> and password file <none>
I20260809 23:11:47.433779 32051 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:47.433849 32051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:47.433991 32051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:47.435202 32051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "db9b9b5451414d5eaa4957543c9f1f64"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "a2d1b15e06757afa9d24bfd5d27c557d"
server_key_iv: "fd81062b942d1a5ca4ef76a0c1ad9909"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.435622 32051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "db9b9b5451414d5eaa4957543c9f1f64"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "a2d1b15e06757afa9d24bfd5d27c557d"
server_key_iv: "fd81062b942d1a5ca4ef76a0c1ad9909"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.437094 32051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:11:47.437989 32074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.438170 32051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:47.438265 32051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "db9b9b5451414d5eaa4957543c9f1f64"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "a2d1b15e06757afa9d24bfd5d27c557d"
server_key_iv: "fd81062b942d1a5ca4ef76a0c1ad9909"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.438351 32051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:47.449096 32051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:47.451257 32051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:47.451438 32051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:47.451718 32051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:47.452064 32051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:47.452117 32051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.452162 32051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:47.452193 32051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.458451 32051 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:41769
I20260809 23:11:47.458495 32257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:41769 every 8 connection(s)
I20260809 23:11:47.458875 32051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:11:47.465174 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 32051
I20260809 23:11:47.465327 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:11:47.465550 8759 external_mini_cluster.cc:1428] Setting key 88fb9b742c5f50d0b70e95fff8567f57
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:47.470060 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.459790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:41891 (local address 127.8.141.254:41297)
0809 23:11:47.459865 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.459868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.460162 (+ 294us) server_negotiation.cc:407] Connection header received
0809 23:11:47.460681 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.460682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.460699 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.460760 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:47.461888 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.462417 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.463466 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.463668 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.463741 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.466479 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:47.466480 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:47.466480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:47.466488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:47.468254 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.468826 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.468827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.468828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.468886 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.469203 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.469205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.469205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.469337 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:47.469462 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.469801 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.469915 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:11:47.470084 32260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.459788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:41891)
0809 23:11:47.460021 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.460063 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.460584 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.460838 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.460841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.461583 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:47.461784 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.461790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.462530 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.462531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.463375 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.463377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.463813 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.463814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.463950 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.464532 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:47.464542 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:47.466384 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:47.468357 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:47.468367 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:47.468370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:47.468726 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:47.468953 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:47.468955 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:47.468955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:47.469111 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:47.469473 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:47.469480 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:47.469651 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.469753 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.469905 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":122,"threads_started":1}
I20260809 23:11:47.470626 32258 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41297
I20260809 23:11:47.470746 32258 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:47.470961 32258 heartbeater.cc:511] Master 127.8.141.254:41297 requested a full tablet report, sending...
I20260809 23:11:47.471382 31736 ts_manager.cc:194] Registered new tserver with Master: db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769)
I20260809 23:11:47.471787 31736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:41891
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:11:47.497308 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41297
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:47.587857 32264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:47.588100 32264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:47.588127 32264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:47.589414 32264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:47.589473 32264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:47.589504 32264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:11:47.591331 32264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:47.591573 32264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:47.591792 32264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:47.598153 32277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:47.598179 32264 server_base.cc:1034] running on GCE node
W20260809 23:11:47.598385 32274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.598464 32272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:47.598820 32264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:47.599120 32264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:47.600291 32264 hybrid_clock.cc:648] HybridClock initialized: now 1786317107600255 us; error 56 us; skew 500 ppm
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:47.603291 32264 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:11:47.603798 32264 webserver.cc:460] Webserver started at http://127.8.141.195:34815/ using document root <none> and password file <none>
I20260809 23:11:47.604045 32264 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:47.604111 32264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:47.604274 32264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:11:47.605481 32264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "ac11e45b178745c1a7323cbdfc1bf646"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "5c2ab7f4f0ad09066509cf63c680964a"
server_key_iv: "ab6a50b40ad43238d60a397c7285a13e"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.606030 32264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "ac11e45b178745c1a7323cbdfc1bf646"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "5c2ab7f4f0ad09066509cf63c680964a"
server_key_iv: "ab6a50b40ad43238d60a397c7285a13e"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.607551 32264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:11:47.608500 32291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.608728 32264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:47.608843 32264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "ac11e45b178745c1a7323cbdfc1bf646"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "5c2ab7f4f0ad09066509cf63c680964a"
server_key_iv: "ab6a50b40ad43238d60a397c7285a13e"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.608935 32264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:47.624640 32264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:47.627243 32264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:47.627421 32264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:47.627694 32264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:47.628051 32264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:47.628106 32264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.628147 32264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:47.628188 32264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.634805 32264 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:41665
I20260809 23:11:47.634876 32486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:41665 every 8 connection(s)
I20260809 23:11:47.635236 32264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:47.644071 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 32264
I20260809 23:11:47.644160 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:11:47.644351 8759 external_mini_cluster.cc:1428] Setting key 76009ddeda87232c4f23e549ecaabc60
I20260809 23:11:47.646080 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.636045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:52003 (local address 127.8.141.254:41297)
0809 23:11:47.636135 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.636137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.636302 (+ 165us) server_negotiation.cc:407] Connection header received
0809 23:11:47.636806 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.636807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.636818 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.636941 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:47.638083 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.638614 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.639643 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.639848 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.639915 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.642746 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:47.642754 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:47.642754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:47.642763 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:47.644451 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.644912 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.644913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.644914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.644987 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.645270 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.645271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.645271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.645477 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:47.645561 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.645869 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.645969 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 23:11:47.646333 32490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.635998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:52003)
0809 23:11:47.636171 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.636214 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.636722 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.637007 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.637010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.637667 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:47.637873 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.637878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.638717 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.638718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.639545 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.639547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.639948 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.639949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.640066 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.640754 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:47.640764 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:47.642608 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:47.644547 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:47.644557 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:47.644560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:47.644824 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:47.645062 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:47.645063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:47.645063 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:47.645194 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:47.645551 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:47.645560 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:47.645723 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.645846 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.646055 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260809 23:11:47.646852 32487 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41297
I20260809 23:11:47.646973 32487 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:47.647198 32487 heartbeater.cc:511] Master 127.8.141.254:41297 requested a full tablet report, sending...
I20260809 23:11:47.647652 31736 ts_manager.cc:194] Registered new tserver with Master: ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
I20260809 23:11:47.648034 31736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:52003
I20260809 23:11:47.656184 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:11:47.664652 31804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.657831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47192 (local address 127.8.141.254:41297)
0809 23:11:47.657900 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.657902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.658053 (+ 151us) server_negotiation.cc:407] Connection header received
0809 23:11:47.658138 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.658140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.658151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.658211 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:47.658849 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.659364 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.660123 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.660334 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.660423 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.661728 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:47.661730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:47.661730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:47.661739 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:47.663282 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.663784 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.663785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.663786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.663855 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:47.664059 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:47.664060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:47.664060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:47.664231 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:47.664348 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.664447 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.664539 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:11:47.666445 31736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:11:47.668002 31736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:11:47.674980 32508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.670988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.8.141.194:41769)
0809 23:11:47.671225 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.671229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.671240 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:11:47.671282 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.671285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.671304 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.671377 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:47.671587 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.672735 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674029 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.674528 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674616 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.674683 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.674852 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.674896 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
I20260809 23:11:47.674968 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.670647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41769 (local address 127.0.0.1:35054)
0809 23:11:47.670779 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.670826 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.670895 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.671402 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.671405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.671407 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:47.671495 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.671498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.672850 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.672851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.673916 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.673918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.674617 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.674709 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.674735 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.674819 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.674880 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":2,"thread_start_us":98,"threads_started":1}
I20260809 23:11:47.675148 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.670647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:45683 (local address 127.0.0.1:45660)
0809 23:11:47.671176 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.671189 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.671247 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.671720 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.671721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.671723 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:47.671763 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.671764 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.673049 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.673050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.674013 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.674016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.674905 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.674973 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.674978 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.675009 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.675058 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":2,"thread_start_us":77,"threads_started":1}
I20260809 23:11:47.675362 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.670949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.8.141.195:41665)
0809 23:11:47.671174 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.671180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.671460 (+ 280us) server_negotiation.cc:407] Connection header received
0809 23:11:47.671511 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.671513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.671530 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.671603 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:47.671726 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.672413 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674012 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.674781 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674864 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.674926 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.675201 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.675258 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
I20260809 23:11:47.675595 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.670954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.8.141.193:45683)
0809 23:11:47.671544 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.671549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.671560 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:11:47.671599 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.671601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.671617 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.671851 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:47.671931 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.672953 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674112 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.674809 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.675357 (+ 548us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.675431 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.675459 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.675506 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":130,"threads_started":1}
I20260809 23:11:47.675159 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.670652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41665 (local address 127.0.0.1:55782)
0809 23:11:47.671340 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.671349 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.671394 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.671612 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.671613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.671616 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:47.671668 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.671670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.672518 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.672519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.673914 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.673916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.674956 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.674957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.675014 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.675017 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.675063 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.675095 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"spinlock_wait_cycles":768,"thread_start_us":151,"threads_started":1}
I20260809 23:11:47.676328 31944 tablet_service.cc:1467] Processing CreateTablet for tablet 76119d69d25f4079b40795edc691a33e (DEFAULT_TABLE table=test-table [id=500a6c1487e941d0a26b01b9f6963ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:47.676331 32147 tablet_service.cc:1467] Processing CreateTablet for tablet 76119d69d25f4079b40795edc691a33e (DEFAULT_TABLE table=test-table [id=500a6c1487e941d0a26b01b9f6963ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:47.676688 31944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76119d69d25f4079b40795edc691a33e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:47.676688 32147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76119d69d25f4079b40795edc691a33e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:47.678005 32368 tablet_service.cc:1467] Processing CreateTablet for tablet 76119d69d25f4079b40795edc691a33e (DEFAULT_TABLE table=test-table [id=500a6c1487e941d0a26b01b9f6963ac4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:47.678411 32368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76119d69d25f4079b40795edc691a33e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:47.679780 32510 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Bootstrap starting.
I20260809 23:11:47.680223 32512 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap starting.
I20260809 23:11:47.680745 32510 tablet_bootstrap.cc:654] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:47.681037 32512 tablet_bootstrap.cc:654] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:47.681423 32510 log.cc:824] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:47.681663 32512 log.cc:824] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:47.682101 32515 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Bootstrap starting.
I20260809 23:11:47.682305 32510 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: No bootstrap required, opened a new log
I20260809 23:11:47.682402 32510 ts_tablet_manager.cc:1397] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 23:11:47.682756 32515 tablet_bootstrap.cc:654] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:47.683298 32515 log.cc:824] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:47.684226 32515 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: No bootstrap required, opened a new log
I20260809 23:11:47.684303 32515 ts_tablet_manager.cc:1397] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:11:47.684211 32512 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: No bootstrap required, opened a new log
I20260809 23:11:47.684417 32512 ts_tablet_manager.cc:1397] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 23:11:47.684773 32510 raft_consensus.cc:348] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.684964 32510 raft_consensus.cc:374] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:47.685004 32510 raft_consensus.cc:729] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db9b9b5451414d5eaa4957543c9f1f64, State: Initialized, Role: FOLLOWER
I20260809 23:11:47.685177 32510 consensus_queue.cc:260] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.685501 32510 ts_tablet_manager.cc:1428] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:11:47.685662 32258 heartbeater.cc:503] Master 127.8.141.254:41297 was elected leader, sending a full tablet report...
I20260809 23:11:47.686424 32515 raft_consensus.cc:348] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.686601 32515 raft_consensus.cc:374] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:47.686650 32515 raft_consensus.cc:729] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac11e45b178745c1a7323cbdfc1bf646, State: Initialized, Role: FOLLOWER
I20260809 23:11:47.686638 32512 raft_consensus.cc:348] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.686766 32515 consensus_queue.cc:260] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.686848 32512 raft_consensus.cc:374] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:47.686900 32512 raft_consensus.cc:729] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852711c331424839a7bbe8efc0d2ed0b, State: Initialized, Role: FOLLOWER
I20260809 23:11:47.687077 32515 ts_tablet_manager.cc:1428] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:11:47.687072 32512 consensus_queue.cc:260] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.687165 32526 raft_consensus.cc:482] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:47.687209 32487 heartbeater.cc:503] Master 127.8.141.254:41297 was elected leader, sending a full tablet report...
I20260809 23:11:47.687248 32526 raft_consensus.cc:504] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.687625 32512 ts_tablet_manager.cc:1428] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:11:47.687634 32526 leader_election.cc:288] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
I20260809 23:11:47.687758 32044 heartbeater.cc:503] Master 127.8.141.254:41297 was elected leader, sending a full tablet report...
I20260809 23:11:47.691401 32508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.688232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:43403 (local address 127.8.141.194:41769)
0809 23:11:47.688285 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.688287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.688296 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:11:47.688318 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.688318 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.688328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.688381 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:47.688530 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.689287 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.690421 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.690983 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.691048 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.691057 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.691290 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.691335 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:11:47.691419 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.688297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:59081 (local address 127.8.141.195:41665)
0809 23:11:47.688336 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.688338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.688347 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:11:47.688370 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.688371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.688380 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.688463 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:47.688642 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.689393 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.690343 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.690960 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.691046 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.691057 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.691290 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.691335 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:11:47.691421 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.687980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41665 (local address 127.8.141.193:59081)
0809 23:11:47.688150 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.688159 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.688191 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.688477 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.688478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.688480 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:47.688550 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.688551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.689465 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.689466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.690248 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.690251 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.691083 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.691085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.691178 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.691185 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.691258 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.691322 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":9,"thread_start_us":82,"threads_started":1}
I20260809 23:11:47.691428 32529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.687859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41769 (local address 127.8.141.193:43403)
0809 23:11:47.688108 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.688121 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:47.688179 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:47.688387 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:47.688389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:47.688391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:47.688469 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.688471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.689440 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.689441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.690344 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:47.690347 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.691083 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.691085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:47.691178 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.691185 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:11:47.691258 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:11:47.691322 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":32}
I20260809 23:11:47.691778 32406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76119d69d25f4079b40795edc691a33e" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac11e45b178745c1a7323cbdfc1bf646" is_pre_election: true
I20260809 23:11:47.691937 32182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76119d69d25f4079b40795edc691a33e" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9b9b5451414d5eaa4957543c9f1f64" is_pre_election: true
I20260809 23:11:47.691994 32406 raft_consensus.cc:2454] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 0.
I20260809 23:11:47.692101 32182 raft_consensus.cc:2454] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 0.
I20260809 23:11:47.692217 31862 leader_election.cc:302] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b, ac11e45b178745c1a7323cbdfc1bf646; no voters:
I20260809 23:11:47.692364 32528 raft_consensus.cc:2781] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:11:47.692430 32528 raft_consensus.cc:482] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:11:47.692461 32528 raft_consensus.cc:3037] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:47.693184 32528 raft_consensus.cc:504] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.693331 32528 leader_election.cc:288] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 election: Requested vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
I20260809 23:11:47.693507 32406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76119d69d25f4079b40795edc691a33e" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac11e45b178745c1a7323cbdfc1bf646"
I20260809 23:11:47.693517 32182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76119d69d25f4079b40795edc691a33e" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9b9b5451414d5eaa4957543c9f1f64"
I20260809 23:11:47.693584 32406 raft_consensus.cc:3037] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:47.693603 32182 raft_consensus.cc:3037] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:47.694329 32182 raft_consensus.cc:2454] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 1.
I20260809 23:11:47.694509 31861 leader_election.cc:302] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b, db9b9b5451414d5eaa4957543c9f1f64; no voters:
I20260809 23:11:47.694600 32528 raft_consensus.cc:2781] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:11:47.694617 32406 raft_consensus.cc:2454] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 1.
I20260809 23:11:47.694679 32528 raft_consensus.cc:686] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 1 LEADER]: Becoming Leader. State: Replica: 852711c331424839a7bbe8efc0d2ed0b, State: Running, Role: LEADER
I20260809 23:11:47.694785 32528 consensus_queue.cc:237] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:47.695814 31736 catalog_manager.cc:5495] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 852711c331424839a7bbe8efc0d2ed0b (127.8.141.193). New cstate: current_term: 1 leader_uuid: "852711c331424839a7bbe8efc0d2ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } health_report { overall_health: UNKNOWN } } }
I20260809 23:11:47.706185 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.703488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45662 (local address 127.8.141.193:45683)
0809 23:11:47.703580 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:47.703582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:47.703591 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:11:47.703625 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:47.703626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:47.703646 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:47.703715 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:11:47.703925 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.704459 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.705216 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:47.705525 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:47.705593 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:47.705768 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:11:47.705880 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:11:47.705956 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:47.706031 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:11:47.706050 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 23:11:47.708180 32182 raft_consensus.cc:1261] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Refusing update from remote peer 852711c331424839a7bbe8efc0d2ed0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:47.708192 32406 raft_consensus.cc:1261] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Refusing update from remote peer 852711c331424839a7bbe8efc0d2ed0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:47.708534 32528 consensus_queue.cc:1035] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:11:47.708659 32526 consensus_queue.cc:1035] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 23:11:47.709499 32259 tablet.cc:2369] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:11:47.711483 32534 mvcc.cc:205] Tried to move back new op lower bound from 7316754873169629184 to 7316754873118883840. Current Snapshot: MvccSnapshot[applied={T|T < 7316754873169629184}]
I20260809 23:11:47.713662 31736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:11:47.713780 31736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:11:47.714638 31736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Sending DeleteTablet for 3 replicas of tablet 76119d69d25f4079b40795edc691a33e
I20260809 23:11:47.715018 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 31836
I20260809 23:11:47.714991 31944 tablet_service.cc:1514] Processing DeleteTablet for tablet 76119d69d25f4079b40795edc691a33e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:47 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:45660
I20260809 23:11:47.715019 32368 tablet_service.cc:1514] Processing DeleteTablet for tablet 76119d69d25f4079b40795edc691a33e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:47 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:55782
I20260809 23:11:47.715016 32147 tablet_service.cc:1514] Processing DeleteTablet for tablet 76119d69d25f4079b40795edc691a33e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:47 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:35054
I20260809 23:11:47.724570 32549 tablet_replica.cc:330] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: stopping tablet replica
I20260809 23:11:47.724773 32549 raft_consensus.cc:2229] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:11:47.724931 32549 raft_consensus.cc:2258] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:47.725136 32551 tablet_replica.cc:330] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: stopping tablet replica
I20260809 23:11:47.725270 32551 raft_consensus.cc:2229] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:11:47.725405 32551 raft_consensus.cc:2258] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:47.725703 32549 ts_tablet_manager.cc:1905] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:11:47.727340 32551 ts_tablet_manager.cc:1905] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:11:47.727630 32549 ts_tablet_manager.cc:1918] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:11:47.727813 32549 log.cc:1196] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/76119d69d25f4079b40795edc691a33e
I20260809 23:11:47.728291 32549 ts_tablet_manager.cc:1939] T 76119d69d25f4079b40795edc691a33e P db9b9b5451414d5eaa4957543c9f1f64: Deleting consensus metadata
I20260809 23:11:47.728973 31710 catalog_manager.cc:4841] TS db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769): tablet 76119d69d25f4079b40795edc691a33e (table test-table [id=500a6c1487e941d0a26b01b9f6963ac4]) successfully deleted
I20260809 23:11:47.729542 32551 ts_tablet_manager.cc:1918] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:11:47.729638 32551 log.cc:1196] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/76119d69d25f4079b40795edc691a33e
I20260809 23:11:47.729930 32551 ts_tablet_manager.cc:1939] T 76119d69d25f4079b40795edc691a33e P ac11e45b178745c1a7323cbdfc1bf646: Deleting consensus metadata
I20260809 23:11:47.730826 31711 catalog_manager.cc:4841] TS ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665): tablet 76119d69d25f4079b40795edc691a33e (table test-table [id=500a6c1487e941d0a26b01b9f6963ac4]) successfully deleted
W20260809 23:11:47.734051 31709 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:45683 (error 108)
I20260809 23:11:47.734280 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 32051
I20260809 23:11:47.734670 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:47.734534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:45683 (local address 127.0.0.1:45664)
0809 23:11:47.734590 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:47.734626 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:45683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 23:11:47.734943 31709 catalog_manager.cc:4584] TS 852711c331424839a7bbe8efc0d2ed0b (127.8.141.193:45683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 76119d69d25f4079b40795edc691a33e: Network error: Client connection negotiation failed: client connection to 127.8.141.193:45683: connect: Connection refused (error 111)
I20260809 23:11:47.749017 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 32264
I20260809 23:11:47.754662 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 31684
I20260809 23:11:47.772269 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41297
--webserver_interface=127.8.141.254
--webserver_port=33757
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:47.863615 32553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:47.863866 32553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:47.863893 32553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:47.865141 32553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:47.865196 32553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:47.865211 32553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:47.865224 32553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:47.865236 32553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:47.866881 32553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:41297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:47.867159 32553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:47.867426 32553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:47.878036 32561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.878036 32560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:47.878052 32566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:47.878737 32553 server_base.cc:1034] running on GCE node
I20260809 23:11:47.879122 32553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:47.879455 32553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:47.880616 32553 hybrid_clock.cc:648] HybridClock initialized: now 1786317107880579 us; error 60 us; skew 500 ppm
Aug 09 23:11:47 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:47.883422 32553 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:11:47.883962 32553 webserver.cc:460] Webserver started at http://127.8.141.254:33757/ using document root <none> and password file <none>
I20260809 23:11:47.884222 32553 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:47.884307 32553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:47.886291 32553 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:11:47.887166 32577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:47.887431 32553 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:47.887557 32553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "8e9ba9c1aa134e268f37bd8ae09882db"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "736f728f03a4ee0bac922d8927d56523"
server_key_iv: "9030aa7e5cfe67407c79ba2548d6ded1"
server_key_version: "encryptionkey@0"
I20260809 23:11:47.888011 32553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:47.907332 32553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:47.909818 32553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:47.910127 32553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:47.914821 32553 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:41297
I20260809 23:11:47.914907 32658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:41297 every 8 connection(s)
I20260809 23:11:47.915396 32553 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:11:47.917879 32659 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:11:47.918663 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Bootstrap starting.
I20260809 23:11:47.919023 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 32553
I20260809 23:11:47.919306 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:45683
--local_ip_for_outbound_sockets=127.8.141.193
--tserver_master_addrs=127.8.141.254:41297
--webserver_port=37997
--webserver_interface=127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:47.929100 32659 log.cc:824] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:47.931434 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:11:47.931792 32659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Bootstrap complete.
I20260809 23:11:47.934934 32659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.935248 32659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ba9c1aa134e268f37bd8ae09882db, State: Initialized, Role: FOLLOWER
I20260809 23:11:47.935473 32659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.935577 32659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:11:47.935645 32659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:11:47.935702 32659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:11:47.936901 32659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.937038 32659 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9ba9c1aa134e268f37bd8ae09882db; no voters:
I20260809 23:11:47.937263 32659 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:11:47.937309 32663 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:11:47.937474 32663 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 LEADER]: Becoming Leader. State: Replica: 8e9ba9c1aa134e268f37bd8ae09882db, State: Running, Role: LEADER
I20260809 23:11:47.937575 32663 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:47.937613 32659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:47.937882 32664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } } }
I20260809 23:11:47.937989 32664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:47.937883 32663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9ba9c1aa134e268f37bd8ae09882db. Latest consensus state: current_term: 2 leader_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } } }
I20260809 23:11:47.938235 32663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:47.938704 32673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:11:47.939534 32673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:11:47.939636 32673 catalog_manager.cc:1259] Loaded cluster ID: 71ee956d368a4d07938acfe561f22b68
I20260809 23:11:47.939667 32673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:11:47.940199 32673 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 23:11:47.945511 32673 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 23:11:47.948365 32673 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:11:47.948613 32673 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Loaded TSK: 0
I20260809 23:11:47.948941 32673 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:11:48.014026 32661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:48.014289 32661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:48.014344 32661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:48.016181 32661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:48.016264 32661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:48.016315 32661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:11:48.018384 32661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:45683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=37997
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:48.018723 32661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:48.018985 32661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:48.025384 32706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.025607 32709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:48.025770 32661 server_base.cc:1034] running on GCE node
W20260809 23:11:48.025390 32703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:48.026203 32661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:48.026558 32661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:48.027752 32661 hybrid_clock.cc:648] HybridClock initialized: now 1786317108027716 us; error 56 us; skew 500 ppm
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:48.036530 32661 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:11:48.037139 32661 webserver.cc:460] Webserver started at http://127.8.141.193:37997/ using document root <none> and password file <none>
I20260809 23:11:48.037482 32661 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:48.037549 32661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:48.039333 32661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:11:48.040125 32720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.040380 32661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:11:48.040485 32661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "852711c331424839a7bbe8efc0d2ed0b"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "9aca2db98d1df1c6d358054fef081bf7"
server_key_iv: "7f06002815eb7ea95f23ad34fe8e3fc6"
server_key_version: "encryptionkey@0"
I20260809 23:11:48.040896 32661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:48.062968 32661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:48.065590 32661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:48.065775 32661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:48.066062 32661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:48.066614 32733 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:11:48.068639 32661 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:11:48.068724 32661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 23:11:48.068776 32661 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:11:48.069634 32661 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:11:48.069690 32661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:48.069761 32733 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap starting.
I20260809 23:11:48.078475 32661 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:45683
I20260809 23:11:48.078910 32661 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:11:48.081795 442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:45683 every 8 connection(s)
I20260809 23:11:48.083565 32733 log.cc:824] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:48.086735 32733 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:11:48.086825 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 32661
I20260809 23:11:48.087062 32733 tablet_bootstrap.cc:492] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap complete.
I20260809 23:11:48.087126 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:41769
--local_ip_for_outbound_sockets=127.8.141.194
--tserver_master_addrs=127.8.141.254:41297
--webserver_port=39419
--webserver_interface=127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:48.088385 32733 ts_tablet_manager.cc:1397] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.010s
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:48.090325 32733 raft_consensus.cc:348] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:48.090682 32733 raft_consensus.cc:729] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852711c331424839a7bbe8efc0d2ed0b, State: Initialized, Role: FOLLOWER
I20260809 23:11:48.090853 32733 consensus_queue.cc:260] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } }
I20260809 23:11:48.091248 32733 ts_tablet_manager.cc:1428] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Time spent starting tablet: real 0.003s user 0.005s sys 0.001s
I20260809 23:11:48.095620 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.079879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:44225 (local address 127.8.141.254:41297)
0809 23:11:48.084011 (+ 4132us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.084024 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.084082 (+ 58us) server_negotiation.cc:407] Connection header received
0809 23:11:48.084258 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.084267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.084343 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.084516 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:48.085325 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.086265 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.087784 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.088433 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.089117 (+ 684us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.091299 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:48.091304 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:48.091305 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:48.091317 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:48.093618 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.094075 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.094079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.094079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.094184 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.094483 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.094484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.094485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.094737 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:48.094828 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.095145 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.095285 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4054,"thread_start_us":108,"threads_started":1}
I20260809 23:11:48.095619 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.079721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:44225)
0809 23:11:48.079937 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.079999 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.080979 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.084559 (+ 3580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.084564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.084991 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:48.085191 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.085197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.086403 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.086404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.087668 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.087672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.088571 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.088573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.088741 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.089457 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:48.089467 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:48.091172 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:48.093726 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.093734 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.093736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.093983 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.094263 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.094264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.094264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.094402 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.094821 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:48.094828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:48.094971 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.095076 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.095284 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
I20260809 23:11:48.096333 444 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41297
I20260809 23:11:48.096431 444 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:48.096897 444 heartbeater.cc:511] Master 127.8.141.254:41297 requested a full tablet report, sending...
I20260809 23:11:48.098297 32602 ts_manager.cc:194] Registered new tserver with Master: 852711c331424839a7bbe8efc0d2ed0b (127.8.141.193:45683)
I20260809 23:11:48.099573 32602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:44225
I20260809 23:11:48.100574 444 heartbeater.cc:503] Master 127.8.141.254:41297 was elected leader, sending a full tablet report...
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317107, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:11:48.108649 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.098988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:45683 (local address 127.0.0.1:45666)
0809 23:11:48.099231 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.099264 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.099324 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.099543 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.099548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.099937 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:48.100010 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.100014 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.101630 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.101631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.102259 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.102263 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.102928 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.102930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.103001 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.103577 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:48.103582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:48.105224 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:48.107329 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.107337 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.107341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.107598 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.107828 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.107829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.107829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.107939 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.108288 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:48.108305 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:48.108363 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.108451 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.108569 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":108,"threads_started":1}
I20260809 23:11:48.108774 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.099099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.8.141.193:45683)
0809 23:11:48.099319 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.099324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.099338 (+ 14us) server_negotiation.cc:407] Connection header received
0809 23:11:48.099420 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.099423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.099439 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.099533 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:48.100109 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.101516 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.102363 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.102835 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.102915 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.105323 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:48.105333 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:48.105333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:48.105376 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:48.107214 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.107687 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.107690 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.107691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.107759 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.108002 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.108003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.108003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.108196 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:48.108281 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.108515 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.108635 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
I20260809 23:11:48.109171 332 tablet_service.cc:1514] Processing DeleteTablet for tablet 76119d69d25f4079b40795edc691a33e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:47 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:45666
I20260809 23:11:48.109819 456 tablet_replica.cc:330] stopping tablet replica
I20260809 23:11:48.109948 456 raft_consensus.cc:2229] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:11:48.110040 456 raft_consensus.cc:2258] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:48.110373 456 ts_tablet_manager.cc:1905] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:11:48.112610 456 ts_tablet_manager.cc:1918] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:11:48.112704 456 log.cc:1196] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/76119d69d25f4079b40795edc691a33e
I20260809 23:11:48.112954 456 ts_tablet_manager.cc:1939] T 76119d69d25f4079b40795edc691a33e P 852711c331424839a7bbe8efc0d2ed0b: Deleting consensus metadata
I20260809 23:11:48.113577 32582 catalog_manager.cc:4841] TS 852711c331424839a7bbe8efc0d2ed0b (127.8.141.193:45683): tablet 76119d69d25f4079b40795edc691a33e (table test-table [id=500a6c1487e941d0a26b01b9f6963ac4]) successfully deleted
W20260809 23:11:48.183830 450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:48.184124 450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:48.184172 450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:48.185469 450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:48.185541 450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:48.185593 450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:11:48.187220 450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:41769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=39419
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:48.187525 450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:48.187775 450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:48.194259 467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.194253 464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.194252 463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:48.194281 450 server_base.cc:1034] running on GCE node
I20260809 23:11:48.194813 450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:48.195070 450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:48.196230 450 hybrid_clock.cc:648] HybridClock initialized: now 1786317108196215 us; error 31 us; skew 500 ppm
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:48.199178 450 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:11:48.199730 450 webserver.cc:460] Webserver started at http://127.8.141.194:39419/ using document root <none> and password file <none>
I20260809 23:11:48.199982 450 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:48.200074 450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:48.201687 450 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:11:48.202466 474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.202699 450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:48.202818 450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "db9b9b5451414d5eaa4957543c9f1f64"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "a2d1b15e06757afa9d24bfd5d27c557d"
server_key_iv: "fd81062b942d1a5ca4ef76a0c1ad9909"
server_key_version: "encryptionkey@0"
I20260809 23:11:48.203219 450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:48.218575 450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:48.220739 450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:48.220917 450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:48.221186 450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:48.221562 450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:48.221616 450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.221653 450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:48.221680 450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.228363 450 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:41769
I20260809 23:11:48.228413 670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:41769 every 8 connection(s)
I20260809 23:11:48.228797 450 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:48.236902 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 450
I20260809 23:11:48.237210 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:41665
--local_ip_for_outbound_sockets=127.8.141.195
--tserver_master_addrs=127.8.141.254:41297
--webserver_port=34815
--webserver_interface=127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:48.240386 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.229656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49275 (local address 127.8.141.254:41297)
0809 23:11:48.229733 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.229736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.229967 (+ 231us) server_negotiation.cc:407] Connection header received
0809 23:11:48.230565 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.230567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.230583 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.230665 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:48.232119 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.232630 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.233995 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.234213 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.234286 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.237010 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:48.237012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:48.237012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:48.237021 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:48.238836 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.239292 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.239293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.239294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.239361 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.239654 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.239655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.239655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.239789 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:48.239859 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.240182 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.240287 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:11:48.240622 675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.229659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:49275)
0809 23:11:48.229831 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.229880 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.230417 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.230801 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.230807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.231768 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:48.231985 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.231991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.232747 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.232749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.233844 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.233848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.234344 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.234345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.234480 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.235048 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:48.235059 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:48.236848 (+ 1789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:48.238922 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.238936 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.238939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.239210 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.239444 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.239446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.239446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.239579 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.239862 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:48.239872 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:48.240039 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.240147 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.240336 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260809 23:11:48.241194 671 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41297
I20260809 23:11:48.241323 671 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:48.241797 671 heartbeater.cc:511] Master 127.8.141.254:41297 requested a full tablet report, sending...
I20260809 23:11:48.242296 32602 ts_manager.cc:194] Registered new tserver with Master: db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769)
I20260809 23:11:48.242908 32602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:49275
W20260809 23:11:48.329598 676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:48.329831 676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:48.329859 676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:48.331118 676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:48.331157 676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:48.331187 676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:11:48.332757 676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:41665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=34815
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:48.332976 676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:48.333180 676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:48.339141 683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.339246 682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.339352 687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:48.339257 676 server_base.cc:1034] running on GCE node
I20260809 23:11:48.339633 676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:48.340030 676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:48.341219 676 hybrid_clock.cc:648] HybridClock initialized: now 1786317108341178 us; error 60 us; skew 500 ppm
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:48.344134 676 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:11:48.344722 676 webserver.cc:460] Webserver started at http://127.8.141.195:34815/ using document root <none> and password file <none>
I20260809 23:11:48.344967 676 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:48.345028 676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:48.346634 676 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:48.347442 697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.347707 676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:48.347811 676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "ac11e45b178745c1a7323cbdfc1bf646"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "5c2ab7f4f0ad09066509cf63c680964a"
server_key_iv: "ab6a50b40ad43238d60a397c7285a13e"
server_key_version: "encryptionkey@0"
I20260809 23:11:48.348362 676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:48.359295 676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:48.362169 676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:48.362361 676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:48.362646 676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:48.362991 676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:48.363049 676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.363113 676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:48.363144 676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.369864 676 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:41665
I20260809 23:11:48.369974 894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:41665 every 8 connection(s)
I20260809 23:11:48.370453 676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:11:48.374388 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 676
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:11:48.381975 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.371315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:49427 (local address 127.8.141.254:41297)
0809 23:11:48.371401 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.371403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.371632 (+ 229us) server_negotiation.cc:407] Connection header received
0809 23:11:48.372214 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.372215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.372229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.372288 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:48.373300 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.373863 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.375072 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.375398 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.375454 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.378348 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:48.378349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:48.378350 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:48.378357 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:48.380043 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.380663 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.380665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.380665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.380736 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.381082 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.381083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.381084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.381235 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:48.381313 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.381755 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.381859 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 23:11:48.382180 897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.371315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:49427)
0809 23:11:48.371508 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.371547 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.372103 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.372365 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.372370 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.372988 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:48.373195 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.373199 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.373973 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.373974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.374921 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.374924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.375504 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.375506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.375685 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.376367 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:48.376377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:48.378215 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:48.380203 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.380216 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.380219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.380572 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.380857 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:48.380858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:48.380858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:48.381001 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:48.381369 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:48.381379 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:48.381571 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.381701 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.381865 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":102,"threads_started":1}
I20260809 23:11:48.382722 895 heartbeater.cc:347] Connected to a master server at 127.8.141.254:41297
I20260809 23:11:48.382841 895 heartbeater.cc:464] Registering TS with master...
I20260809 23:11:48.382881 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.374922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.8.141.254:41297)
0809 23:11:48.375064 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.375066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.375115 (+ 49us) server_negotiation.cc:407] Connection header received
0809 23:11:48.375193 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.375194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.375207 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.375282 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:48.376160 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.376800 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.377769 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.377976 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.378037 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.379312 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:48.379313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:48.379314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:48.379322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:48.381236 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.381724 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.381725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.381725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.381825 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:48.382182 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:48.382183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:48.382184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:48.382360 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:48.382544 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.382588 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.382678 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
I20260809 23:11:48.383077 895 heartbeater.cc:511] Master 127.8.141.254:41297 requested a full tablet report, sending...
I20260809 23:11:48.383453 32601 ts_manager.cc:194] Registered new tserver with Master: ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
I20260809 23:11:48.383838 32601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:49427
I20260809 23:11:48.394819 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:11:48.398144 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.395283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.8.141.254:41297)
0809 23:11:48.395408 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.395411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.395513 (+ 102us) server_negotiation.cc:407] Connection header received
0809 23:11:48.395628 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.395629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.395675 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.395756 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:11:48.395914 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.396414 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.397231 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.397520 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.397603 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.397786 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:11:48.397907 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:11:48.398006 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.398024 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.398050 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 23:11:48.398537 32601 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47200:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:11:48.400166 32601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:11:48.402375 332 tablet_service.cc:1467] Processing CreateTablet for tablet c32d4eefa67643e3bbf96252d9148b93 (DEFAULT_TABLE table=test-table [id=c37c5da806cd4103957162a25a658f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:48.402592 332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c32d4eefa67643e3bbf96252d9148b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:48.404057 32733 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap starting.
I20260809 23:11:48.404439 32733 tablet_bootstrap.cc:654] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:48.405055 32733 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: No bootstrap required, opened a new log
I20260809 23:11:48.405104 32733 ts_tablet_manager.cc:1397] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:48.405475 32733 raft_consensus.cc:348] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.405565 32733 raft_consensus.cc:374] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:48.405604 32733 raft_consensus.cc:729] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852711c331424839a7bbe8efc0d2ed0b, State: Initialized, Role: FOLLOWER
I20260809 23:11:48.405651 32733 consensus_queue.cc:260] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.405762 32733 ts_tablet_manager.cc:1428] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:48.406267 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.402112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41769 (local address 127.0.0.1:35056)
0809 23:11:48.402257 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.402268 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.402308 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.402716 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.402717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.402719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:48.402769 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.402770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.403840 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.403841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.404874 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.404878 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.405901 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.405902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.406046 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.406070 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.406124 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.406178 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260809 23:11:48.406524 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.402291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.8.141.194:41769)
0809 23:11:48.402535 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.402540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.402551 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:11:48.402595 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.402598 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.402615 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.402698 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:48.402826 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.403744 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.404965 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.405729 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.405801 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.405864 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.406374 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.406426 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":133,"threads_started":1}
I20260809 23:11:48.407282 550 tablet_service.cc:1467] Processing CreateTablet for tablet c32d4eefa67643e3bbf96252d9148b93 (DEFAULT_TABLE table=test-table [id=c37c5da806cd4103957162a25a658f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:48.407676 550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c32d4eefa67643e3bbf96252d9148b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:48.408006 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.402383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55786 (local address 127.8.141.195:41665)
0809 23:11:48.404166 (+ 1783us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.404171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.404183 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:11:48.404230 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.404232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.404249 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.404355 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:48.404446 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.405656 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.407051 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.407680 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.407754 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.407812 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.407909 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.407941 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1740,"thread_start_us":116,"threads_started":1}
I20260809 23:11:48.408010 905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.402112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41665 (local address 127.0.0.1:55786)
0809 23:11:48.402725 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.402737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.402793 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.404348 (+ 1555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.404349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.404351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:48.404391 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.404393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.405792 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.405793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.406948 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.406952 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.407764 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.407766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.407830 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.407834 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.407899 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.407939 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":446,"thread_start_us":68,"threads_started":1}
I20260809 23:11:48.408779 792 tablet_service.cc:1467] Processing CreateTablet for tablet c32d4eefa67643e3bbf96252d9148b93 (DEFAULT_TABLE table=test-table [id=c37c5da806cd4103957162a25a658f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:11:48.409180 792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c32d4eefa67643e3bbf96252d9148b93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:11:48.411757 908 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Bootstrap starting.
I20260809 23:11:48.412431 908 tablet_bootstrap.cc:654] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:48.412770 911 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Bootstrap starting.
I20260809 23:11:48.413033 908 log.cc:824] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:48.413614 911 tablet_bootstrap.cc:654] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Neither blocks nor log segments found. Creating new log.
I20260809 23:11:48.413946 908 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: No bootstrap required, opened a new log
I20260809 23:11:48.414041 908 ts_tablet_manager.cc:1397] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:11:48.414331 911 log.cc:824] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:48.415172 911 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: No bootstrap required, opened a new log
I20260809 23:11:48.415266 911 ts_tablet_manager.cc:1397] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:11:48.415692 908 raft_consensus.cc:348] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.415841 908 raft_consensus.cc:374] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:48.415867 908 raft_consensus.cc:729] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db9b9b5451414d5eaa4957543c9f1f64, State: Initialized, Role: FOLLOWER
I20260809 23:11:48.415987 908 consensus_queue.cc:260] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.416275 908 ts_tablet_manager.cc:1428] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:11:48.416306 671 heartbeater.cc:503] Master 127.8.141.254:41297 was elected leader, sending a full tablet report...
I20260809 23:11:48.417289 911 raft_consensus.cc:348] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.417533 911 raft_consensus.cc:374] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:11:48.417577 911 raft_consensus.cc:729] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac11e45b178745c1a7323cbdfc1bf646, State: Initialized, Role: FOLLOWER
I20260809 23:11:48.417707 911 consensus_queue.cc:260] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.417995 911 ts_tablet_manager.cc:1428] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:11:48.418040 895 heartbeater.cc:503] Master 127.8.141.254:41297 was elected leader, sending a full tablet report...
W20260809 23:11:48.479444 673 tablet.cc:2369] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:11:48.550520 451 raft_consensus.cc:482] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:48.550717 451 raft_consensus.cc:504] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.551092 451 leader_election.cc:288] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
I20260809 23:11:48.555699 903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.551494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:43951 (local address 127.8.141.194:41769)
0809 23:11:48.551545 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.551547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.551556 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:11:48.551593 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.551594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.551604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.551695 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:48.551869 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.552731 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.554330 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.555013 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.555116 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.555127 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.555503 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.555608 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 23:11:48.555699 904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.551443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:51485 (local address 127.8.141.195:41665)
0809 23:11:48.551517 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.551520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.551663 (+ 143us) server_negotiation.cc:407] Connection header received
0809 23:11:48.551789 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.551790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.551799 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.551879 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:11:48.552024 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.552829 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.554330 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.555027 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.555116 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.555127 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.555503 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.555627 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 23:11:48.555779 925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.551325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41665 (local address 127.8.141.193:51485)
0809 23:11:48.551592 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.551607 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.551651 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.551901 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.551902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.551903 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:48.551962 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.551963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.552915 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.552915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.554204 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.554208 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.555215 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.555216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.555309 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.555365 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.555470 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.555650 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":160,"thread_start_us":77,"threads_started":1}
I20260809 23:11:48.556001 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.551325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41769 (local address 127.8.141.193:43951)
0809 23:11:48.551397 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.551410 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:48.551467 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:48.551707 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:48.551708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:48.551710 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:11:48.551793 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.551795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.552815 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.552816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.554204 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:48.554208 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.555176 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.555179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:48.555326 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.555365 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 23:11:48.555470 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 23:11:48.555738 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":64}
I20260809 23:11:48.556252 824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c32d4eefa67643e3bbf96252d9148b93" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac11e45b178745c1a7323cbdfc1bf646" is_pre_election: true
I20260809 23:11:48.556348 585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c32d4eefa67643e3bbf96252d9148b93" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9b9b5451414d5eaa4957543c9f1f64" is_pre_election: true
I20260809 23:11:48.556458 824 raft_consensus.cc:2454] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 0.
I20260809 23:11:48.556519 585 raft_consensus.cc:2454] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 0.
I20260809 23:11:48.556746 32723 leader_election.cc:302] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b, ac11e45b178745c1a7323cbdfc1bf646; no voters:
I20260809 23:11:48.556916 451 raft_consensus.cc:2781] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:11:48.557010 451 raft_consensus.cc:482] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:11:48.557055 451 raft_consensus.cc:3037] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:48.558106 451 raft_consensus.cc:504] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.558305 451 leader_election.cc:288] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 election: Requested vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
I20260809 23:11:48.558473 585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c32d4eefa67643e3bbf96252d9148b93" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db9b9b5451414d5eaa4957543c9f1f64"
I20260809 23:11:48.558560 585 raft_consensus.cc:3037] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:48.559368 585 raft_consensus.cc:2454] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 1.
I20260809 23:11:48.559508 824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c32d4eefa67643e3bbf96252d9148b93" candidate_uuid: "852711c331424839a7bbe8efc0d2ed0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac11e45b178745c1a7323cbdfc1bf646"
I20260809 23:11:48.559598 824 raft_consensus.cc:3037] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:11:48.559677 32722 leader_election.cc:302] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b, db9b9b5451414d5eaa4957543c9f1f64; no voters:
I20260809 23:11:48.559792 451 raft_consensus.cc:2781] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:11:48.559974 451 raft_consensus.cc:686] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 LEADER]: Becoming Leader. State: Replica: 852711c331424839a7bbe8efc0d2ed0b, State: Running, Role: LEADER
I20260809 23:11:48.560063 451 consensus_queue.cc:237] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:48.560595 824 raft_consensus.cc:2454] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 852711c331424839a7bbe8efc0d2ed0b in term 1.
I20260809 23:11:48.561035 32601 catalog_manager.cc:5495] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 852711c331424839a7bbe8efc0d2ed0b (127.8.141.193). New cstate: current_term: 1 leader_uuid: "852711c331424839a7bbe8efc0d2ed0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } health_report { overall_health: HEALTHY } } }
W20260809 23:11:48.582656 445 tablet.cc:2369] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:11:48.591882 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.589063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45670 (local address 127.8.141.193:45683)
0809 23:11:48.589140 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:48.589142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:48.589151 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:11:48.589202 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:48.589203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:48.589225 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:48.589306 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:11:48.589457 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.589977 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.591041 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:48.591278 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:48.591359 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:48.591521 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:11:48.591634 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:11:48.591717 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:48.591761 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 23:11:48.591785 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 23:11:48.593750 585 raft_consensus.cc:1261] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Refusing update from remote peer 852711c331424839a7bbe8efc0d2ed0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:48.593863 824 raft_consensus.cc:1261] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Refusing update from remote peer 852711c331424839a7bbe8efc0d2ed0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:11:48.594252 927 consensus_queue.cc:1035] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:11:48.594403 451 consensus_queue.cc:1035] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:11:48.596771 933 mvcc.cc:205] Tried to move back new op lower bound from 7316754876796456960 to 7316754876662611968. Current Snapshot: MvccSnapshot[applied={T|T < 7316754876796456960}]
I20260809 23:11:48.597003 935 mvcc.cc:205] Tried to move back new op lower bound from 7316754876796456960 to 7316754876662611968. Current Snapshot: MvccSnapshot[applied={T|T < 7316754876796456960}]
I20260809 23:11:48.597961 934 mvcc.cc:205] Tried to move back new op lower bound from 7316754876796456960 to 7316754876662611968. Current Snapshot: MvccSnapshot[applied={T|T < 7316754876796456960}]
I20260809 23:11:48.600065 32601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:11:48.600185 32601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47200:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:11:48.601449 32601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Sending DeleteTablet for 3 replicas of tablet c32d4eefa67643e3bbf96252d9148b93
I20260809 23:11:48.601850 550 tablet_service.cc:1514] Processing DeleteTablet for tablet c32d4eefa67643e3bbf96252d9148b93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:48 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:35056
I20260809 23:11:48.601979 332 tablet_service.cc:1514] Processing DeleteTablet for tablet c32d4eefa67643e3bbf96252d9148b93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:48 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:45666
I20260809 23:11:48.601992 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 32661
I20260809 23:11:48.602072 456 tablet_replica.cc:330] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: stopping tablet replica
I20260809 23:11:48.602124 456 raft_consensus.cc:2229] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:11:48.602128 792 tablet_service.cc:1514] Processing DeleteTablet for tablet c32d4eefa67643e3bbf96252d9148b93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:11:48 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:55786
I20260809 23:11:48.602223 456 raft_consensus.cc:2258] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:48.616990 949 tablet_replica.cc:330] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: stopping tablet replica
I20260809 23:11:48.617211 949 raft_consensus.cc:2229] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:11:48.617395 950 tablet_replica.cc:330] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: stopping tablet replica
I20260809 23:11:48.617408 949 raft_consensus.cc:2258] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:48.617681 950 raft_consensus.cc:2229] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:11:48.617883 950 raft_consensus.cc:2258] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:11:48.619236 949 ts_tablet_manager.cc:1905] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:11:48.621762 949 ts_tablet_manager.cc:1918] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:11:48.621876 949 log.cc:1196] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/c32d4eefa67643e3bbf96252d9148b93
I20260809 23:11:48.622288 949 ts_tablet_manager.cc:1939] T c32d4eefa67643e3bbf96252d9148b93 P db9b9b5451414d5eaa4957543c9f1f64: Deleting consensus metadata
I20260809 23:11:48.623220 32584 catalog_manager.cc:4841] TS db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769): tablet c32d4eefa67643e3bbf96252d9148b93 (table test-table [id=c37c5da806cd4103957162a25a658f56]) successfully deleted
I20260809 23:11:48.623517 950 ts_tablet_manager.cc:1905] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:11:48.625612 950 ts_tablet_manager.cc:1918] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:11:48.625711 950 log.cc:1196] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/c32d4eefa67643e3bbf96252d9148b93
I20260809 23:11:48.626013 950 ts_tablet_manager.cc:1939] T c32d4eefa67643e3bbf96252d9148b93 P ac11e45b178745c1a7323cbdfc1bf646: Deleting consensus metadata
I20260809 23:11:48.626569 32585 catalog_manager.cc:4841] TS ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665): tablet c32d4eefa67643e3bbf96252d9148b93 (table test-table [id=c37c5da806cd4103957162a25a658f56]) successfully deleted
W20260809 23:11:48.629447 32582 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:45683 (error 108)
I20260809 23:11:48.629613 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 450
I20260809 23:11:48.630163 905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:48.630016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:45683 (local address 127.0.0.1:45676)
0809 23:11:48.630074 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:48.630108 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:45683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 23:11:48.630283 32582 catalog_manager.cc:4584] TS 852711c331424839a7bbe8efc0d2ed0b (127.8.141.193:45683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c32d4eefa67643e3bbf96252d9148b93: Network error: Client connection negotiation failed: client connection to 127.8.141.193:45683: connect: Connection refused (error 111)
I20260809 23:11:48.648451 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 676
I20260809 23:11:48.670186 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 32553
I20260809 23:11:48.687965 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41297
--webserver_interface=127.8.141.254
--webserver_port=33757
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:41297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:48.779078 955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:48.779378 955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:48.779426 955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:48.780773 955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:11:48.780853 955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:48.780879 955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:48.780900 955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:11:48.780927 955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:11:48.782550 955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:41297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=33757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:48.782866 955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:48.783118 955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:48.788849 967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:48.788949 955 server_base.cc:1034] running on GCE node
W20260809 23:11:48.788885 969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.788919 966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:48.789816 955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:48.790179 955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:48.791381 955 hybrid_clock.cc:648] HybridClock initialized: now 1786317108791361 us; error 39 us; skew 500 ppm
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:48.794569 955 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:11:48.795337 955 webserver.cc:460] Webserver started at http://127.8.141.254:33757/ using document root <none> and password file <none>
I20260809 23:11:48.795640 955 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:48.795718 955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:48.797727 955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:11:48.798731 980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.798998 955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:48.799122 955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "8e9ba9c1aa134e268f37bd8ae09882db"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "736f728f03a4ee0bac922d8927d56523"
server_key_iv: "9030aa7e5cfe67407c79ba2548d6ded1"
server_key_version: "encryptionkey@0"
I20260809 23:11:48.799713 955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:48.826484 955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:48.828941 955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:48.829082 955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:48.833537 955 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:41297
I20260809 23:11:48.833575 1065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:41297 every 8 connection(s)
I20260809 23:11:48.833997 955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:11:48.834546 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 955
I20260809 23:11:48.834847 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:45683
--local_ip_for_outbound_sockets=127.8.141.193
--tserver_master_addrs=127.8.141.254:41297
--webserver_port=37997
--webserver_interface=127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:48.835867 1066 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:11:48.836758 1066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Bootstrap starting.
I20260809 23:11:48.849838 1066 log.cc:824] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:48.852842 1066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:11:48.853193 1066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db: Bootstrap complete.
I20260809 23:11:48.855695 1066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:48.855947 1066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ba9c1aa134e268f37bd8ae09882db, State: Initialized, Role: FOLLOWER
I20260809 23:11:48.856210 1066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:48.856292 1066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:11:48.856354 1066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:11:48.856405 1066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:11:48.857612 1066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:48.857734 1066 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9ba9c1aa134e268f37bd8ae09882db; no voters:
I20260809 23:11:48.857950 1066 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 23:11:48.857993 1075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 3 FOLLOWER]: Leader election won for term 3
I20260809 23:11:48.858112 1075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [term 3 LEADER]: Becoming Leader. State: Replica: 8e9ba9c1aa134e268f37bd8ae09882db, State: Running, Role: LEADER
I20260809 23:11:48.858217 1075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } }
I20260809 23:11:48.858256 1066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:11:48.858491 1075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9ba9c1aa134e268f37bd8ae09882db. Latest consensus state: current_term: 3 leader_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } } }
I20260809 23:11:48.858569 1075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:48.858498 1076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ba9c1aa134e268f37bd8ae09882db" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 41297 } } }
I20260809 23:11:48.858798 1085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:11:48.858821 1076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ba9c1aa134e268f37bd8ae09882db [sys.catalog]: This master's current role is: LEADER
I20260809 23:11:48.859726 1085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:11:48.859845 1085 catalog_manager.cc:1259] Loaded cluster ID: 71ee956d368a4d07938acfe561f22b68
I20260809 23:11:48.859881 1085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 23:11:48.860245 1085 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f52e0d7f9ad google::LogMessage::Fail() at ??:0
@ 0x7f52e0d8288a google::LogMessage::SendToLog() at ??:0
@ 0x7f52e0d7f4de google::LogMessage::Flush() at ??:0
@ 0x7f52e0d80a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f52e8ca00f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f52e8ca98b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f52e2c4d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f52e2c42954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f52e3aa66db start_thread at ??:0
@ 0x7f52df87d61f clone at ??:0
W20260809 23:11:48.952960 1071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:48.953248 1071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:48.953298 1071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:48.954968 1071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:48.955032 1071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:48.955065 1071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:11:48.957376 1071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:45683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=37997
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:48.957661 1071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:48.957883 1071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:48.963712 1138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.963914 1140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:48.963712 1143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:48.964048 1071 server_base.cc:1034] running on GCE node
I20260809 23:11:48.964323 1071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:48.964622 1071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:48.965806 1071 hybrid_clock.cc:648] HybridClock initialized: now 1786317108965777 us; error 40 us; skew 500 ppm
Aug 09 23:11:48 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:48.969444 1071 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:11:48.970010 1071 webserver.cc:460] Webserver started at http://127.8.141.193:37997/ using document root <none> and password file <none>
I20260809 23:11:48.970254 1071 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:48.970305 1071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:48.971802 1071 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:48.972512 1153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:48.972744 1071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:48.972889 1071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "852711c331424839a7bbe8efc0d2ed0b"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "9aca2db98d1df1c6d358054fef081bf7"
server_key_iv: "7f06002815eb7ea95f23ad34fe8e3fc6"
server_key_version: "encryptionkey@0"
I20260809 23:11:48.973402 1071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:48.995436 1071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:48.997874 1071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:48.998025 1071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:48.998319 1071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:48.998835 1169 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:11:49.001031 1071 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:11:49.001130 1071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 23:11:49.001184 1071 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:11:49.002104 1071 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:11:49.002184 1071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:49.002256 1169 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap starting.
I20260809 23:11:49.010152 1071 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:45683
I20260809 23:11:49.010248 1351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:45683 every 8 connection(s)
I20260809 23:11:49.010643 1071 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:11:49.012784 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 1071
I20260809 23:11:49.013070 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:41769
--local_ip_for_outbound_sockets=127.8.141.194
--tserver_master_addrs=127.8.141.254:41297
--webserver_port=39419
--webserver_interface=127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:11:49.014611 1169 log.cc:824] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:11:49.017730 1169 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:11:49.018087 1169 tablet_bootstrap.cc:492] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Bootstrap complete.
I20260809 23:11:49.019588 1169 ts_tablet_manager.cc:1397] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.010s
I20260809 23:11:49.021834 1169 raft_consensus.cc:348] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:49.022307 1169 raft_consensus.cc:729] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 852711c331424839a7bbe8efc0d2ed0b, State: Initialized, Role: FOLLOWER
I20260809 23:11:49.022509 1169 consensus_queue.cc:260] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:49.022995 1169 ts_tablet_manager.cc:1428] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260809 23:11:49.127624 1361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:49.127959 1361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:49.128003 1361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:49.129652 1361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:49.129740 1361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:49.129787 1361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:11:49.131908 1361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:41769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=39419
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:49.132151 1361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:49.132369 1361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:49.138320 1374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:49.138343 1373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:49.138324 1377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:49.138710 1361 server_base.cc:1034] running on GCE node
I20260809 23:11:49.138965 1361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:49.139261 1361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:49.140483 1361 hybrid_clock.cc:648] HybridClock initialized: now 1786317109140430 us; error 69 us; skew 500 ppm
Aug 09 23:11:49 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317109, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:49.144938 1361 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:11:49.145527 1361 webserver.cc:460] Webserver started at http://127.8.141.194:39419/ using document root <none> and password file <none>
I20260809 23:11:49.145823 1361 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:49.145916 1361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:49.147576 1361 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:11:49.148362 1389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:49.148557 1361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:11:49.148669 1361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "db9b9b5451414d5eaa4957543c9f1f64"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "a2d1b15e06757afa9d24bfd5d27c557d"
server_key_iv: "fd81062b942d1a5ca4ef76a0c1ad9909"
server_key_version: "encryptionkey@0"
I20260809 23:11:49.149477 1361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:49.165046 1361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:49.167520 1361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:49.167675 1361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:49.167938 1361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:49.168265 1361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:49.168301 1361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:49.168325 1361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:49.168365 1361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:49.174575 1361 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:41769
I20260809 23:11:49.174722 1570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:41769 every 8 connection(s)
I20260809 23:11:49.175087 1361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:11:49.182839 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 1361
I20260809 23:11:49.183234 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:41665
--local_ip_for_outbound_sockets=127.8.141.195
--tserver_master_addrs=127.8.141.254:41297
--webserver_port=34815
--webserver_interface=127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:11:49.286425 1577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:11:49.286718 1577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:11:49.286765 1577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:11:49.288237 1577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:11:49.288334 1577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:11:49.288395 1577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:11:49.290246 1577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:41665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=34815
--tserver_master_addrs=127.8.141.254:41297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:11:49.290542 1577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:11:49.290845 1577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:11:49.296887 1584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:49.296885 1583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:11:49.297089 1587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:11:49.297411 1577 server_base.cc:1034] running on GCE node
I20260809 23:11:49.297639 1577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:11:49.297950 1577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:11:49.299149 1577 hybrid_clock.cc:648] HybridClock initialized: now 1786317109299134 us; error 42 us; skew 500 ppm
Aug 09 23:11:49 dist-test-slave-j2vl krb5kdc[31666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317109, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:11:49.302469 1577 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:11:49.303059 1577 webserver.cc:460] Webserver started at http://127.8.141.195:34815/ using document root <none> and password file <none>
I20260809 23:11:49.303269 1577 fs_manager.cc:359] Metadata directory not provided
I20260809 23:11:49.303314 1577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:11:49.305089 1577 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:11:49.305825 1597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:49.306051 1577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:11:49.306138 1577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "ac11e45b178745c1a7323cbdfc1bf646"
format_stamp: "Formatted at 2026-08-09 23:11:47 on dist-test-slave-j2vl"
server_key: "5c2ab7f4f0ad09066509cf63c680964a"
server_key_iv: "ab6a50b40ad43238d60a397c7285a13e"
server_key_version: "encryptionkey@0"
I20260809 23:11:49.306497 1577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:11:49.334498 1577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:11:49.337041 1577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:11:49.337209 1577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:11:49.337507 1577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:11:49.337857 1577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:11:49.337911 1577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:49.337956 1577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:11:49.337993 1577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:11:49.350136 1577 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:41665
I20260809 23:11:49.350181 1794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:41665 every 8 connection(s)
I20260809 23:11:49.350581 1577 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
W20260809 23:11:49.351572 1360 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:11:49.011402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:50879)
0809 23:11:49.011591 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.011641 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:49.012414 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:49.351366 (+338952us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
W20260809 23:11:49.351608 1575 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:11:49.175947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:49159)
0809 23:11:49.176147 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.176197 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:49.176869 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:49.351385 (+174516us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
W20260809 23:11:49.351814 1353 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:41297 (0 consecutive failures): Network error: Failed to ping master at 127.8.141.254:41297: Client connection negotiation failed: client connection to 127.8.141.254:41297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:11:49.351963 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:49.351354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:45765)
0809 23:11:49.351700 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.351794 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":92,"threads_started":1}
W20260809 23:11:49.351995 1572 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:41297 (0 consecutive failures): Network error: Failed to ping master at 127.8.141.254:41297: Client connection negotiation failed: client connection to 127.8.141.254:41297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:11:49.352113 1360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:49.351994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:44563)
0809 23:11:49.352047 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.352089 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 23:11:49.352331 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:49.352221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:33437)
0809 23:11:49.352270 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.352309 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 23:11:49.352433 1795 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:41297 (0 consecutive failures): Network error: Failed to ping master at 127.8.141.254:41297: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
I20260809 23:11:49.352519 1360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:49.352432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:53375)
0809 23:11:49.352474 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.352499 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 23:11:49.352649 1575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:49.352571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:38179)
0809 23:11:49.352606 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.352628 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 23:11:49.352669 1353 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:11:49.352775 1572 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:11:49.352859 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:49.352756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:47461)
0809 23:11:49.352795 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.352833 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 23:11:49.353108 1801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:49.353060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:55279)
0809 23:11:49.353080 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:49.353097 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 23:11:49.353224 1795 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:11:49.353555 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 1577
I20260809 23:11:50.191716 1805 raft_consensus.cc:482] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:50.191864 1805 raft_consensus.cc:504] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:50.192310 1805 leader_election.cc:288] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
Aug 09 23:11:50 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.195@KRBTEST.COM
Aug 09 23:11:50 dist-test-slave-j2vl krb5kdc[31666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317108, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.194@KRBTEST.COM
I20260809 23:11:50.202286 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:50.192727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:38945 (local address 127.8.141.195:41665)
0809 23:11:50.192872 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:50.192888 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:50.192906 (+ 18us) server_negotiation.cc:407] Connection header received
0809 23:11:50.193395 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:50.193398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:50.193458 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:50.193594 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:50.193937 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.194819 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.195664 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.196068 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.196192 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:50.198645 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:50.198650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:50.198650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:50.198664 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:50.200504 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:50.200960 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:50.200963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:50.200964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:50.201063 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:50.201369 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:50.201370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:50.201370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:50.201637 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:50.201732 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:50.202039 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 23:11:50.202173 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260809 23:11:50.202332 1807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:50.192525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41665 (local address 127.8.141.193:38945)
0809 23:11:50.192650 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:50.192663 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:50.192737 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:50.193582 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:50.193583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:50.193769 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:50.193809 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:50.193811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.194919 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.194920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:50.195574 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:50.195575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.196161 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.196162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:50.196278 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:50.196992 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:50.196998 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:50.198528 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:50.200601 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:50.200609 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:50.200612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:50.200861 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:50.201138 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:50.201139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:50.201139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:50.201274 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:50.201722 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:50.201729 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:50.201867 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 23:11:50.201982 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 23:11:50.202176 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 23:11:50.202416 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:50.192644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:35241 (local address 127.8.141.194:41769)
0809 23:11:50.192819 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 23:11:50.192825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:11:50.192837 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:11:50.192938 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:11:50.192939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:11:50.192959 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:11:50.193066 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:11:50.193778 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.194679 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.195669 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.196077 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.196193 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:50.199127 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:11:50.199132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:11:50.199132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:11:50.199145 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:11:50.200885 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:50.201278 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:50.201281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:50.201282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:50.201441 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:11:50.201657 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:11:50.201658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:11:50.201658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:11:50.201911 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:11:50.202003 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 23:11:50.202168 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 23:11:50.202302 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":56,"threads_started":1}
I20260809 23:11:50.202507 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:50.192527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41769 (local address 127.8.141.193:35241)
0809 23:11:50.192680 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:50.192692 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:11:50.192751 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:11:50.193108 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:11:50.193111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:11:50.193457 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:11:50.193647 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:50.193654 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.194807 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.194808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:50.195576 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:11:50.195578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:11:50.196170 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:11:50.196171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:11:50.196279 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:11:50.196936 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:11:50.196945 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:11:50.199027 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:11:50.200994 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:50.200995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:50.200996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:50.201192 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:50.201510 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:11:50.201511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:11:50.201512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:11:50.201584 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:11:50.201986 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:11:50.201987 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:11:50.202035 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 23:11:50.202103 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:11:50.202246 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":36,"thread_start_us":56,"threads_started":1}
W20260809 23:11:50.203287 1163 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
W20260809 23:11:50.203385 1161 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
I20260809 23:11:50.203451 1161 leader_election.cc:302] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b; no voters: ac11e45b178745c1a7323cbdfc1bf646, db9b9b5451414d5eaa4957543c9f1f64
I20260809 23:11:50.203562 1805 raft_consensus.cc:2726] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:11:50.353309 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:50.353145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:49755)
0809 23:11:50.353216 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:50.353261 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 23:11:50.353643 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:50.353144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:59395)
0809 23:11:50.353401 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:50.353581 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":103,"threads_started":1}
I20260809 23:11:50.353899 1814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:50.353597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:48131)
0809 23:11:50.353785 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:50.353835 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":63,"threads_started":1}
I20260809 23:11:51.354463 1815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:51.354202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:36317)
0809 23:11:51.354360 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:51.354406 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260809 23:11:51.354549 1818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:51.354345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:36037)
0809 23:11:51.354481 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:51.354509 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260809 23:11:51.354528 1817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:51.354293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:53087)
0809 23:11:51.354431 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:51.354475 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260809 23:11:52.004925 1819 raft_consensus.cc:482] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:52.005031 1819 raft_consensus.cc:504] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:52.005221 1819 leader_election.cc:288] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
W20260809 23:11:52.005687 1163 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
W20260809 23:11:52.005795 1161 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
I20260809 23:11:52.005884 1161 leader_election.cc:302] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b; no voters: ac11e45b178745c1a7323cbdfc1bf646, db9b9b5451414d5eaa4957543c9f1f64
I20260809 23:11:52.005982 1819 raft_consensus.cc:2726] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:11:52.355322 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:52.355050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:55267)
0809 23:11:52.355222 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:52.355281 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
I20260809 23:11:52.355324 1826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:52.355091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:48897)
0809 23:11:52.355212 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:52.355253 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":52,"threads_started":1}
I20260809 23:11:52.355301 1822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:52.355049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:56587)
0809 23:11:52.355199 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:52.355245 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260809 23:11:53.356189 1830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:53.355930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:43695)
0809 23:11:53.356083 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:53.356146 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260809 23:11:53.356184 1833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:53.355940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:48195)
0809 23:11:53.356107 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:53.356138 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":55,"threads_started":1}
I20260809 23:11:53.356185 1832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:53.355918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:34619)
0809 23:11:53.356086 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:53.356127 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260809 23:11:53.596369 1835 raft_consensus.cc:482] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:53.596485 1835 raft_consensus.cc:504] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:53.596683 1835 leader_election.cc:288] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
W20260809 23:11:53.597226 1163 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
W20260809 23:11:53.597375 1161 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
I20260809 23:11:53.597436 1161 leader_election.cc:302] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b; no voters: ac11e45b178745c1a7323cbdfc1bf646, db9b9b5451414d5eaa4957543c9f1f64
I20260809 23:11:53.597509 1835 raft_consensus.cc:2726] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:11:54.357049 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:54.356801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:47793)
0809 23:11:54.356964 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:54.357000 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260809 23:11:54.357080 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:54.356801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:57087)
0809 23:11:54.356974 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:54.357022 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
I20260809 23:11:54.357197 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:54.356968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:56695)
0809 23:11:54.357118 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:54.357151 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260809 23:11:55.358004 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:55.357724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:52565)
0809 23:11:55.357910 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:55.357958 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260809 23:11:55.358013 1843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:55.357766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:42467)
0809 23:11:55.357907 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:55.357951 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260809 23:11:55.358008 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:55.357724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:39829)
0809 23:11:55.357906 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:55.357950 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":90,"threads_started":1}
I20260809 23:11:56.358826 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:56.358569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:50747)
0809 23:11:56.358727 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:56.358768 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":75,"threads_started":1}
I20260809 23:11:56.358805 1845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:56.358555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:50421)
0809 23:11:56.358701 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:56.358751 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260809 23:11:56.358904 1848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:56.358670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:51403)
0809 23:11:56.358821 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:56.358856 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260809 23:11:56.814113 1849 raft_consensus.cc:482] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:11:56.814232 1849 raft_consensus.cc:504] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:11:56.814473 1849 leader_election.cc:288] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
W20260809 23:11:56.814998 1161 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
W20260809 23:11:56.815063 1163 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
I20260809 23:11:56.815204 1163 leader_election.cc:302] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b; no voters: ac11e45b178745c1a7323cbdfc1bf646, db9b9b5451414d5eaa4957543c9f1f64
I20260809 23:11:56.815317 1849 raft_consensus.cc:2726] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:11:57.359654 1850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:57.359389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:38901)
0809 23:11:57.359557 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:57.359606 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260809 23:11:57.359676 1853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:57.359429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:51313)
0809 23:11:57.359588 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:57.359628 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
I20260809 23:11:57.359676 1855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:57.359490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:49193)
0809 23:11:57.359598 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:57.359628 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":49,"threads_started":1}
I20260809 23:11:58.360492 1856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:58.360216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:49093)
0809 23:11:58.360388 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:58.360445 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260809 23:11:58.360492 1858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:58.360236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:52633)
0809 23:11:58.360397 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:58.360439 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":73,"threads_started":1}
I20260809 23:11:58.360492 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:58.360256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:57135)
0809 23:11:58.360398 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:58.360439 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260809 23:11:59.361333 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:59.361106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:45999)
0809 23:11:59.361243 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:59.361281 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260809 23:11:59.361333 1870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:59.361106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:39833)
0809 23:11:59.361243 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:59.361281 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260809 23:11:59.361332 1866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:11:59.361072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:41105)
0809 23:11:59.361245 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:11:59.361281 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":106,"threads_started":1}
I20260809 23:12:00.362155 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:00.361951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:60393)
0809 23:12:00.362077 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:00.362113 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":51,"threads_started":1}
I20260809 23:12:00.362155 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:00.361897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:54861)
0809 23:12:00.362044 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:00.362093 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260809 23:12:00.362155 1875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:00.361890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:55357)
0809 23:12:00.362081 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:00.362114 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":84,"threads_started":1}
I20260809 23:12:01.363092 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:01.362891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:33743)
0809 23:12:01.363018 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:01.363049 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
I20260809 23:12:01.363085 1877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:01.362823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:46013)
0809 23:12:01.362975 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:01.363024 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 23:12:01.363092 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:01.362867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:37923)
0809 23:12:01.363019 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:01.363054 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260809 23:12:01.551368 1880 raft_consensus.cc:482] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:12:01.551496 1880 raft_consensus.cc:504] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db9b9b5451414d5eaa4957543c9f1f64" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41769 } } peers { permanent_uuid: "ac11e45b178745c1a7323cbdfc1bf646" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41665 } } peers { permanent_uuid: "852711c331424839a7bbe8efc0d2ed0b" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 45683 } }
I20260809 23:12:01.552052 1880 leader_election.cc:288] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769), ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665)
W20260809 23:12:01.552608 1163 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ac11e45b178745c1a7323cbdfc1bf646 (127.8.141.195:41665): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
W20260809 23:12:01.552693 1161 leader_election.cc:341] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer db9b9b5451414d5eaa4957543c9f1f64 (127.8.141.194:41769): Not found: Tablet not found: c32d4eefa67643e3bbf96252d9148b93
I20260809 23:12:01.552728 1161 leader_election.cc:302] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 852711c331424839a7bbe8efc0d2ed0b; no voters: ac11e45b178745c1a7323cbdfc1bf646, db9b9b5451414d5eaa4957543c9f1f64
I20260809 23:12:01.552816 1880 raft_consensus.cc:2726] T c32d4eefa67643e3bbf96252d9148b93 P 852711c331424839a7bbe8efc0d2ed0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:12:02.363979 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:02.363715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:38493)
0809 23:12:02.363879 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:02.363928 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260809 23:12:02.363988 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:02.363718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:53689)
0809 23:12:02.363908 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:02.363943 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":79,"threads_started":1}
I20260809 23:12:02.364256 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:02.363732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:60219)
0809 23:12:02.364163 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:02.364207 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":64,"threads_started":1}
I20260809 23:12:03.364948 1889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:03.364626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:42859)
0809 23:12:03.364833 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:03.364881 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":91,"threads_started":1}
I20260809 23:12:03.365083 1890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:03.364832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.193:36637)
0809 23:12:03.364981 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:03.365027 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260809 23:12:03.365123 1891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:03.364856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:40089)
0809 23:12:03.365033 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:03.365071 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":76,"threads_started":1}
I20260809 23:12:04.362346 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 1071
I20260809 23:12:04.369838 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:04.367856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.194:46423)
0809 23:12:04.369724 (+ 1868us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:04.369773 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1846,"thread_start_us":1777,"threads_started":1}
I20260809 23:12:04.370371 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:04.367686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:41297 (local address 127.8.141.195:33935)
0809 23:12:04.370270 (+ 2584us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:04.370315 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2560,"thread_start_us":60,"threads_started":1}
I20260809 23:12:04.375232 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 1361
I20260809 23:12:04.391420 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 1577
2026-08-09T23:12:04Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18953 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:12:04 dist-test-slave-j2vl krb5kdc[1898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:12:04 dist-test-slave-j2vl krb5kdc[1898](info): set up 2 sockets
Aug 09 23:12:04 dist-test-slave-j2vl krb5kdc[1898](info): commencing operation
krb5kdc: starting...
W20260809 23:12:05.710763 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:12:05 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:12:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:12:05.861123 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42437
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:12:05.949709 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:05.949990 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:05.950037 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:05.951372 1918 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:12:05.951427 1918 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:12:05.951442 1918 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:12:05.951453 1918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:12:05.951467 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:05.951480 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:05.951493 1918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:12:05.951505 1918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:12:05.953169 1918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:05.953510 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:05.953799 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:05.959440 1930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:05.959712 1929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:05.959769 1935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:05.960278 1918 server_base.cc:1034] running on GCE node
I20260809 23:12:05.960557 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:05.960915 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:05.962090 1918 hybrid_clock.cc:648] HybridClock initialized: now 1786317125962000 us; error 135 us; skew 500 ppm
Aug 09 23:12:05 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317125, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:05.965059 1918 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:12:05.965770 1918 webserver.cc:460] Webserver started at http://127.8.141.254:33267/ using document root <none> and password file <none>
I20260809 23:12:05.966063 1918 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:05.966142 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:05.966300 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:12:05.967828 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "53f5878cb7f94ab0bde29da00fbc1c41"
format_stamp: "Formatted at 2026-08-09 23:12:05 on dist-test-slave-j2vl"
server_key: "3469eaeb40fb8d96c2a9dd9a7c4ac89f"
server_key_iv: "113f76df257fe94e7329a8553a45fb71"
server_key_version: "encryptionkey@0"
I20260809 23:12:05.968267 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "53f5878cb7f94ab0bde29da00fbc1c41"
format_stamp: "Formatted at 2026-08-09 23:12:05 on dist-test-slave-j2vl"
server_key: "3469eaeb40fb8d96c2a9dd9a7c4ac89f"
server_key_iv: "113f76df257fe94e7329a8553a45fb71"
server_key_version: "encryptionkey@0"
I20260809 23:12:05.969846 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 23:12:05.970745 1947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:05.970990 1918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:05.971108 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "53f5878cb7f94ab0bde29da00fbc1c41"
format_stamp: "Formatted at 2026-08-09 23:12:05 on dist-test-slave-j2vl"
server_key: "3469eaeb40fb8d96c2a9dd9a7c4ac89f"
server_key_iv: "113f76df257fe94e7329a8553a45fb71"
server_key_version: "encryptionkey@0"
I20260809 23:12:05.971192 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:05.985877 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:05.989040 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:05.989223 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:05.993957 1918 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42437
I20260809 23:12:05.994027 2037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42437 every 8 connection(s)
I20260809 23:12:05.994376 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:12:05.995379 2038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:12:05.997872 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 1918
I20260809 23:12:05.997984 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:12:05.998108 2038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Bootstrap starting.
I20260809 23:12:05.998184 8759 external_mini_cluster.cc:1428] Setting key 1e43c0c16ad1a7bce883f7b05660e2b5
I20260809 23:12:05.998896 2038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Neither blocks nor log segments found. Creating new log.
I20260809 23:12:05.999233 2038 log.cc:824] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:05.999970 2038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: No bootstrap required, opened a new log
I20260809 23:12:06.001981 2038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:06.002151 2038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:12:06.002215 2038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f5878cb7f94ab0bde29da00fbc1c41, State: Initialized, Role: FOLLOWER
I20260809 23:12:06.002385 2038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:06.002470 2038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:12:06.002511 2038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:12:06.002560 2038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:12:06.003196 2038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:06.003319 2038 leader_election.cc:302] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f5878cb7f94ab0bde29da00fbc1c41; no voters:
I20260809 23:12:06.003793 2038 leader_election.cc:288] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:12:06.003830 2050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:12:06.004024 2050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 LEADER]: Becoming Leader. State: Replica: 53f5878cb7f94ab0bde29da00fbc1c41, State: Running, Role: LEADER
I20260809 23:12:06.004124 2050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:06.004170 2038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:12:06.004494 2050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } } }
I20260809 23:12:06.004674 2050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: This master's current role is: LEADER
I20260809 23:12:06.004884 2050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f5878cb7f94ab0bde29da00fbc1c41. Latest consensus state: current_term: 1 leader_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } } }
I20260809 23:12:06.005021 2050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: This master's current role is: LEADER
I20260809 23:12:06.005406 2059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 23:12:06 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:12:06.006070 2059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:12:06.007812 2059 catalog_manager.cc:1347] Generated new cluster ID: eddc596b3d124ed8b36a874efee927b4
I20260809 23:12:06.007874 2059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:12:06.010474 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:05.998760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.8.141.254:42437)
0809 23:12:05.998981 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:05.998992 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:05.999025 (+ 33us) server_negotiation.cc:407] Connection header received
0809 23:12:05.999517 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:05.999525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:05.999584 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:05.999731 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:06.000464 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.001437 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.002208 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.002775 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.002891 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.006558 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:06.006562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:06.006563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:06.006577 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:06.008641 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.009139 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.009141 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.009142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.009246 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.009515 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.009515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.009516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.009823 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:06.009975 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.010109 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.010244 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260809 23:12:06.038064 2059 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:12:06.039013 2059 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:12:06.049562 2059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Generated new TSK 0
I20260809 23:12:06.049815 2059 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:12:06.090350 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42437
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:12:06.180554 2079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:06.180820 2079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:06.180863 2079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:06.182394 2079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:06.182461 2079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:06.182505 2079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:12:06.184331 2079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:06.184544 2079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:06.184743 2079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:06.190198 2085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:06.190501 2090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:06.190208 2084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:06.190587 2079 server_base.cc:1034] running on GCE node
I20260809 23:12:06.190809 2079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:06.191056 2079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:06.192392 2079 hybrid_clock.cc:648] HybridClock initialized: now 1786317126192362 us; error 34 us; skew 500 ppm
Aug 09 23:12:06 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:06.195461 2079 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:12:06.196044 2079 webserver.cc:460] Webserver started at http://127.8.141.193:42889/ using document root <none> and password file <none>
I20260809 23:12:06.196323 2079 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:06.196398 2079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:06.196557 2079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:12:06.197751 2079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "76b6aaa363b64a55a30b1d9979c76e35"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "c61d1085ef597edf56459cb807143a49"
server_key_iv: "3812068746d66105d989d041e0a021f8"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.198135 2079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "76b6aaa363b64a55a30b1d9979c76e35"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "c61d1085ef597edf56459cb807143a49"
server_key_iv: "3812068746d66105d989d041e0a021f8"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.199755 2079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:12:06.201046 2100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.201417 2079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:06.201562 2079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "76b6aaa363b64a55a30b1d9979c76e35"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "c61d1085ef597edf56459cb807143a49"
server_key_iv: "3812068746d66105d989d041e0a021f8"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.201658 2079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:06.226758 2079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:06.229326 2079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:06.229558 2079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:06.229864 2079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:06.230249 2079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:12:06.230314 2079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.230578 2079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:12:06.230716 2079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.237795 2079 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:37135
I20260809 23:12:06.237828 2300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:37135 every 8 connection(s)
I20260809 23:12:06.238277 2079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:12:06 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:12:06.247727 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 2079
I20260809 23:12:06.247839 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:12:06.248104 8759 external_mini_cluster.cc:1428] Setting key ec373aafc57354f57c6fb6922d3e1063
I20260809 23:12:06.250196 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.239224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:51893 (local address 127.8.141.254:42437)
0809 23:12:06.239359 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.239362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.239556 (+ 194us) server_negotiation.cc:407] Connection header received
0809 23:12:06.240738 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.240739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.240754 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.240816 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:06.241990 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.242571 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.243646 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.243887 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.243972 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.246449 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:06.246450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:06.246450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:06.246466 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:06.248561 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.249064 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.249065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.249066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.249137 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.249459 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.249460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.249460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.249593 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:06.249675 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.249990 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.250097 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 23:12:06.250412 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.239133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:51893)
0809 23:12:06.239404 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.239462 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.240597 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.240909 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.240920 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.241648 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:06.241877 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.241882 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.242690 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.242691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.243541 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.243544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.243984 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.243986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.244126 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.244724 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:06.244734 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:06.246335 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:06.248648 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:06.248660 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:06.248663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:06.248995 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:06.249214 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:06.249215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:06.249216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:06.249398 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:06.249665 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:06.249674 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:06.249845 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.249957 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.250142 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":105,"threads_started":1}
I20260809 23:12:06.250962 2301 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42437
I20260809 23:12:06.251092 2301 heartbeater.cc:464] Registering TS with master...
I20260809 23:12:06.251309 2301 heartbeater.cc:511] Master 127.8.141.254:42437 requested a full tablet report, sending...
I20260809 23:12:06.251957 1973 ts_manager.cc:194] Registered new tserver with Master: 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193:37135)
I20260809 23:12:06.252923 1973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:51893
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:12:06.282421 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42437
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:12:06.376116 2308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:06.376346 2308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:06.376375 2308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:06.377799 2308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:06.377868 2308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:06.377899 2308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:12:06.379772 2308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:06.380013 2308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:06.380244 2308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:06.386062 2323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:06.386078 2319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:06.386078 2320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:06.386489 2308 server_base.cc:1034] running on GCE node
I20260809 23:12:06.386780 2308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:06.387079 2308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:06.388299 2308 hybrid_clock.cc:648] HybridClock initialized: now 1786317126388301 us; error 46 us; skew 500 ppm
Aug 09 23:12:06 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:06.391197 2308 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:12:06.391726 2308 webserver.cc:460] Webserver started at http://127.8.141.194:33853/ using document root <none> and password file <none>
I20260809 23:12:06.391999 2308 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:06.392048 2308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:06.392191 2308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:12:06.393278 2308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "d1c6fb36dbb3458285da323cf8e442a8"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "752160e9907f530444e43806e511abdc"
server_key_iv: "d4034d275280c92121501c4e5bc892b9"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.393728 2308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "d1c6fb36dbb3458285da323cf8e442a8"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "752160e9907f530444e43806e511abdc"
server_key_iv: "d4034d275280c92121501c4e5bc892b9"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.395340 2308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:12:06.396241 2335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.396524 2308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:06.396641 2308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "d1c6fb36dbb3458285da323cf8e442a8"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "752160e9907f530444e43806e511abdc"
server_key_iv: "d4034d275280c92121501c4e5bc892b9"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.396744 2308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:06.412040 2308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:06.414755 2308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:06.414963 2308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:06.415241 2308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:06.415597 2308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:12:06.415635 2308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.415659 2308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:12:06.415700 2308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.421813 2308 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:37845
I20260809 23:12:06.421955 2523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:37845 every 8 connection(s)
I20260809 23:12:06.422366 2308 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:12:06 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:12:06.429637 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 2308
I20260809 23:12:06.429754 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:12:06.430090 8759 external_mini_cluster.cc:1428] Setting key 5f0b4ac3ba55792e6ece122ccf3b81f6
I20260809 23:12:06.433540 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.423410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:51951 (local address 127.8.141.254:42437)
0809 23:12:06.423525 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.423528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.423686 (+ 158us) server_negotiation.cc:407] Connection header received
0809 23:12:06.424386 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.424387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.424400 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.424460 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:06.425462 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.426001 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.427058 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.427256 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.427323 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.430018 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:06.430020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:06.430020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:06.430035 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:06.431584 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.432175 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.432177 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.432177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.432251 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.432598 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.432603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.432603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.432795 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:06.432922 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.433214 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.433321 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 23:12:06.433755 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.423345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:51951)
0809 23:12:06.423553 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.423593 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.424235 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.424515 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.424519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.425135 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:06.425377 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.425383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.426115 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.426116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.426948 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.426950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.427354 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.427355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.427561 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.428239 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:06.428249 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:06.429901 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:06.431680 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:06.431691 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:06.431694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:06.432058 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:06.432331 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:06.432332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:06.432333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:06.432499 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:06.432920 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:06.432935 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:06.433071 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.433164 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.433303 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":117,"threads_started":1}
I20260809 23:12:06.434393 2525 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42437
I20260809 23:12:06.434525 2525 heartbeater.cc:464] Registering TS with master...
I20260809 23:12:06.434881 2525 heartbeater.cc:511] Master 127.8.141.254:42437 requested a full tablet report, sending...
I20260809 23:12:06.435384 1973 ts_manager.cc:194] Registered new tserver with Master: d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
I20260809 23:12:06.436121 1973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:51951
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:12:06.470353 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42437
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:12:06.561158 2533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:06.561479 2533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:06.561513 2533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:06.562724 2533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:06.562772 2533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:06.562799 2533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:12:06.564333 2533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:06.564607 2533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:06.564888 2533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:06.570407 2541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:06.570395 2543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:06.570395 2540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:06.570847 2533 server_base.cc:1034] running on GCE node
I20260809 23:12:06.571004 2533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:06.571249 2533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:06.572405 2533 hybrid_clock.cc:648] HybridClock initialized: now 1786317126572392 us; error 29 us; skew 500 ppm
Aug 09 23:12:06 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:06.575254 2533 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:12:06.575755 2533 webserver.cc:460] Webserver started at http://127.8.141.195:46473/ using document root <none> and password file <none>
I20260809 23:12:06.575975 2533 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:06.576020 2533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:06.576112 2533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:12:06.577122 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "1a500275092543c68b29f97e1878e2df"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "94a047cd718e64c042dece2ba9f7fc7b"
server_key_iv: "b0021f8e2c7a4f4ce6d2956f39ee9a56"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.577498 2533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "1a500275092543c68b29f97e1878e2df"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "94a047cd718e64c042dece2ba9f7fc7b"
server_key_iv: "b0021f8e2c7a4f4ce6d2956f39ee9a56"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.578946 2533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:12:06.579751 2555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.580004 2533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:12:06.580125 2533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "1a500275092543c68b29f97e1878e2df"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "94a047cd718e64c042dece2ba9f7fc7b"
server_key_iv: "b0021f8e2c7a4f4ce6d2956f39ee9a56"
server_key_version: "encryptionkey@0"
I20260809 23:12:06.580209 2533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:06.592710 2533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:06.595109 2533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:06.595297 2533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:06.595559 2533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:06.595911 2533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:12:06.595978 2533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.596024 2533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:12:06.596058 2533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:06.602548 2533 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:39669
I20260809 23:12:06.602594 2730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:39669 every 8 connection(s)
I20260809 23:12:06.602938 2533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:12:06.608227 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 2533
I20260809 23:12:06.608453 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:12:06.608631 8759 external_mini_cluster.cc:1428] Setting key be8a6de75ba44eea68f4e40183ddd651
Aug 09 23:12:06 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317126, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:12:06.613457 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.603683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:47997 (local address 127.8.141.254:42437)
0809 23:12:06.603773 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.603776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.603964 (+ 188us) server_negotiation.cc:407] Connection header received
0809 23:12:06.604699 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.604701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.604715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.604774 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:06.605892 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.606420 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.607451 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.607651 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.607739 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.610462 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:06.610464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:06.610464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:06.610473 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:06.611948 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.612407 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.612408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.612409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.612473 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.612747 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.612748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.612749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.612873 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:06.612959 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.613249 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.613332 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 23:12:06.613684 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.603638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:47997)
0809 23:12:06.603827 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.603877 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.604570 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.604841 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.604845 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.605593 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:06.605791 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.605796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.606522 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.606523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.607333 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.607336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.607754 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.607755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.607892 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.608486 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:06.608498 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:06.610355 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:06.612075 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:06.612084 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:06.612087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:06.612325 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:06.612543 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:06.612544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:06.612544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:06.612669 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:06.612948 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:06.612956 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:06.613108 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.613216 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.613431 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260809 23:12:06.614197 2731 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42437
I20260809 23:12:06.614285 2731 heartbeater.cc:464] Registering TS with master...
I20260809 23:12:06.614440 2731 heartbeater.cc:511] Master 127.8.141.254:42437 requested a full tablet report, sending...
I20260809 23:12:06.614785 1973 ts_manager.cc:194] Registered new tserver with Master: 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669)
I20260809 23:12:06.615188 1973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:47997
I20260809 23:12:06.620644 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:12:06.628886 2048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.622111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33018 (local address 127.8.141.254:42437)
0809 23:12:06.622184 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.622186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.622289 (+ 103us) server_negotiation.cc:407] Connection header received
0809 23:12:06.622332 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.622333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.622345 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.622397 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:06.623043 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.623523 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.624291 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.624495 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.624565 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.626009 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:06.626011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:06.626011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:06.626018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:06.627603 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.627954 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.627955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.627955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.628014 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:06.628253 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:06.628254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:06.628254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:06.628420 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:06.628508 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.628658 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.628728 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 23:12:06.630815 1973 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33018:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:12:06.632480 1973 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:12:06.639948 2752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.635529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49602 (local address 127.8.141.195:39669)
0809 23:12:06.635712 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.635718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.636004 (+ 286us) server_negotiation.cc:407] Connection header received
0809 23:12:06.636057 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.636059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.636077 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.636378 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:06.636453 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.637583 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.639007 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.639527 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.639622 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.639691 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.639806 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.639848 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260809 23:12:06.639950 2751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.635321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:39669 (local address 127.0.0.1:49602)
0809 23:12:06.635743 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.635783 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.635881 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.636187 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.636191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.636193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:06.636295 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.636297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.637697 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.637698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.638911 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.638914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.639623 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.639624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.639707 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.639722 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.639793 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.639848 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":6,"thread_start_us":80,"threads_started":1}
I20260809 23:12:06.640213 2753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.635620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56262 (local address 127.8.141.193:37135)
0809 23:12:06.635835 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.635853 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.635973 (+ 120us) server_negotiation.cc:407] Connection header received
0809 23:12:06.636019 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.636022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.636039 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.636115 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:06.636385 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.637099 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.638782 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.639594 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.639673 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.639732 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.640029 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.640100 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":98,"threads_started":1}
I20260809 23:12:06.640547 2748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.635332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37845 (local address 127.0.0.1:55692)
0809 23:12:06.635997 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.636006 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.636057 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.636260 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.636260 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.636262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:06.636290 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.636290 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.637728 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.637728 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.638942 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.638945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.640185 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.640186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.640253 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.640259 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.640395 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.640439 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":1,"thread_start_us":111,"threads_started":1}
I20260809 23:12:06.640703 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.635620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55692 (local address 127.8.141.194:37845)
0809 23:12:06.635945 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.635949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.636123 (+ 174us) server_negotiation.cc:407] Connection header received
0809 23:12:06.636169 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.636171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.636186 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.636264 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:06.636344 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.637584 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.639055 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.639862 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.639934 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.639988 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.640541 (+ 553us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.640601 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":102,"threads_started":1}
I20260809 23:12:06.640764 2628 tablet_service.cc:1467] Processing CreateTablet for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd (DEFAULT_TABLE table=test-table [id=8e9c079ee99240fc8ea061bb1c400fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:12:06.640792 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.635321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37135 (local address 127.0.0.1:56262)
0809 23:12:06.635814 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.635826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.635897 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.636198 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.636199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.636202 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:06.636281 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.636283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.637210 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.637212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.638678 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.638682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.639775 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.639776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.639865 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.639869 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.639925 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.639975 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":35,"thread_start_us":116,"threads_started":1}
I20260809 23:12:06.641216 2628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:12:06.641716 2411 tablet_service.cc:1467] Processing CreateTablet for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd (DEFAULT_TABLE table=test-table [id=8e9c079ee99240fc8ea061bb1c400fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:12:06.641801 2188 tablet_service.cc:1467] Processing CreateTablet for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd (DEFAULT_TABLE table=test-table [id=8e9c079ee99240fc8ea061bb1c400fd1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:12:06.642061 2411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:12:06.642187 2188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:12:06.645058 2756 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap starting.
I20260809 23:12:06.645324 2757 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Bootstrap starting.
I20260809 23:12:06.645673 2762 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Bootstrap starting.
I20260809 23:12:06.646128 2757 tablet_bootstrap.cc:654] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Neither blocks nor log segments found. Creating new log.
I20260809 23:12:06.646303 2762 tablet_bootstrap.cc:654] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Neither blocks nor log segments found. Creating new log.
I20260809 23:12:06.646780 2757 log.cc:824] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:06.646812 2762 log.cc:824] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:06.647742 2762 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: No bootstrap required, opened a new log
I20260809 23:12:06.647820 2756 tablet_bootstrap.cc:654] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Neither blocks nor log segments found. Creating new log.
I20260809 23:12:06.647836 2762 ts_tablet_manager.cc:1397] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:12:06.648409 2756 log.cc:824] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:06.649298 2756 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: No bootstrap required, opened a new log
I20260809 23:12:06.649418 2756 ts_tablet_manager.cc:1397] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 23:12:06.650117 2762 raft_consensus.cc:348] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.650319 2762 raft_consensus.cc:374] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:12:06.650362 2762 raft_consensus.cc:729] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c6fb36dbb3458285da323cf8e442a8, State: Initialized, Role: FOLLOWER
I20260809 23:12:06.650493 2762 consensus_queue.cc:260] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.650805 2762 ts_tablet_manager.cc:1428] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:12:06.650918 2525 heartbeater.cc:503] Master 127.8.141.254:42437 was elected leader, sending a full tablet report...
I20260809 23:12:06.651670 2757 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: No bootstrap required, opened a new log
I20260809 23:12:06.651772 2757 ts_tablet_manager.cc:1397] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260809 23:12:06.651678 2756 raft_consensus.cc:348] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.651855 2756 raft_consensus.cc:374] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:12:06.651906 2756 raft_consensus.cc:729] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b6aaa363b64a55a30b1d9979c76e35, State: Initialized, Role: FOLLOWER
I20260809 23:12:06.652070 2756 consensus_queue.cc:260] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.652372 2756 ts_tablet_manager.cc:1428] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:12:06.652406 2301 heartbeater.cc:503] Master 127.8.141.254:42437 was elected leader, sending a full tablet report...
I20260809 23:12:06.653841 2757 raft_consensus.cc:348] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.653982 2757 raft_consensus.cc:374] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:12:06.654004 2757 raft_consensus.cc:729] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a500275092543c68b29f97e1878e2df, State: Initialized, Role: FOLLOWER
I20260809 23:12:06.654124 2757 consensus_queue.cc:260] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.654428 2757 ts_tablet_manager.cc:1428] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 23:12:06.654567 2731 heartbeater.cc:503] Master 127.8.141.254:42437 was elected leader, sending a full tablet report...
W20260809 23:12:06.673034 2526 tablet.cc:2369] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:12:06.739180 2303 tablet.cc:2369] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:12:06.766850 2768 raft_consensus.cc:482] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:12:06.767009 2768 raft_consensus.cc:504] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.767503 2768 leader_election.cc:288] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669), d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
I20260809 23:12:06.770810 2752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.767806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:52043 (local address 127.8.141.195:39669)
0809 23:12:06.767862 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.767864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.768002 (+ 138us) server_negotiation.cc:407] Connection header received
0809 23:12:06.768083 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.768085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.768094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.768150 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:06.768323 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.768835 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.769886 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.770296 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.770354 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.770362 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.770686 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.770733 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:12:06.770824 2755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.767759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:45521 (local address 127.8.141.194:37845)
0809 23:12:06.767811 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.767814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.768037 (+ 223us) server_negotiation.cc:407] Connection header received
0809 23:12:06.768080 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.768082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.768096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.768154 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:06.768400 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.768856 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.769942 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.770370 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.770524 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.770542 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.770702 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.770750 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:12:06.770839 2774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.767682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37845 (local address 127.8.141.193:45521)
0809 23:12:06.767925 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.767934 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.768022 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.768170 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.768170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.768172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:06.768304 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.768307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.768974 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.768975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.769814 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.769816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.770519 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.770520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.770580 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.770585 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.770690 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.770731 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":51,"thread_start_us":115,"threads_started":1}
I20260809 23:12:06.770852 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.767682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:39669 (local address 127.8.141.193:52043)
0809 23:12:06.767894 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.767906 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:06.767990 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:06.768152 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:06.768153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:06.768155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:06.768239 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.768252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.768922 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.768923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.769789 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:06.769791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.770381 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.770382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:06.770567 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.770574 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:12:06.770650 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 23:12:06.770709 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":141,"threads_started":1}
I20260809 23:12:06.771314 2447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8cc2d2efa6479b90d86abb26c8dcfd" candidate_uuid: "76b6aaa363b64a55a30b1d9979c76e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c6fb36dbb3458285da323cf8e442a8" is_pre_election: true
I20260809 23:12:06.771365 2660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8cc2d2efa6479b90d86abb26c8dcfd" candidate_uuid: "76b6aaa363b64a55a30b1d9979c76e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a500275092543c68b29f97e1878e2df" is_pre_election: true
I20260809 23:12:06.771507 2447 raft_consensus.cc:2454] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76b6aaa363b64a55a30b1d9979c76e35 in term 0.
I20260809 23:12:06.771564 2660 raft_consensus.cc:2454] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76b6aaa363b64a55a30b1d9979c76e35 in term 0.
I20260809 23:12:06.771781 2102 leader_election.cc:302] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76b6aaa363b64a55a30b1d9979c76e35, d1c6fb36dbb3458285da323cf8e442a8; no voters:
I20260809 23:12:06.771962 2768 raft_consensus.cc:2781] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:12:06.772028 2768 raft_consensus.cc:482] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:12:06.772046 2768 raft_consensus.cc:3037] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:12:06.772819 2768 raft_consensus.cc:504] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.773044 2768 leader_election.cc:288] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 1 election: Requested vote from peers 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669), d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
I20260809 23:12:06.773141 2660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8cc2d2efa6479b90d86abb26c8dcfd" candidate_uuid: "76b6aaa363b64a55a30b1d9979c76e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a500275092543c68b29f97e1878e2df"
I20260809 23:12:06.773238 2660 raft_consensus.cc:3037] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:12:06.773242 2447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c8cc2d2efa6479b90d86abb26c8dcfd" candidate_uuid: "76b6aaa363b64a55a30b1d9979c76e35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c6fb36dbb3458285da323cf8e442a8"
I20260809 23:12:06.773331 2447 raft_consensus.cc:3037] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:12:06.774106 2660 raft_consensus.cc:2454] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76b6aaa363b64a55a30b1d9979c76e35 in term 1.
I20260809 23:12:06.774114 2447 raft_consensus.cc:2454] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76b6aaa363b64a55a30b1d9979c76e35 in term 1.
I20260809 23:12:06.774312 2103 leader_election.cc:302] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a500275092543c68b29f97e1878e2df, 76b6aaa363b64a55a30b1d9979c76e35; no voters:
I20260809 23:12:06.774425 2768 raft_consensus.cc:2781] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:12:06.774653 2768 raft_consensus.cc:686] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 LEADER]: Becoming Leader. State: Replica: 76b6aaa363b64a55a30b1d9979c76e35, State: Running, Role: LEADER
I20260809 23:12:06.774780 2768 consensus_queue.cc:237] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:06.775688 1972 catalog_manager.cc:5495] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193). New cstate: current_term: 1 leader_uuid: "76b6aaa363b64a55a30b1d9979c76e35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } health_report { overall_health: UNKNOWN } } }
I20260809 23:12:06.787945 2753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.784846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.8.141.193:37135)
0809 23:12:06.784935 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:06.784937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:06.784961 (+ 24us) server_negotiation.cc:407] Connection header received
0809 23:12:06.785069 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:06.785071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:06.785110 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:06.785191 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:12:06.785488 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.786048 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.786984 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:06.787240 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:06.787303 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:06.787554 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:12:06.787680 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:12:06.787765 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:06.787787 (+ 22us) server_negotiation.cc:299] Negotiation successful
0809 23:12:06.787808 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:12:06.790743 2660 raft_consensus.cc:1261] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 1 FOLLOWER]: Refusing update from remote peer 76b6aaa363b64a55a30b1d9979c76e35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:12:06.790743 2447 raft_consensus.cc:1261] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Refusing update from remote peer 76b6aaa363b64a55a30b1d9979c76e35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:12:06.791190 2776 consensus_queue.cc:1035] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:12:06.791370 2768 consensus_queue.cc:1035] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:12:06.794963 2784 mvcc.cc:205] Tried to move back new op lower bound from 7316754951329861632 to 7316754951270129664. Current Snapshot: MvccSnapshot[applied={T|T < 7316754951329861632 or (T in {7316754951329861632})}]
I20260809 23:12:06.797915 1973 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:12:06.798050 1973 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33018:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:12:06.799053 1973 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Sending DeleteTablet for 3 replicas of tablet 6c8cc2d2efa6479b90d86abb26c8dcfd
I20260809 23:12:06.799540 2188 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:12:06 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:56262
I20260809 23:12:06.799644 2628 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:12:06 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:49602
I20260809 23:12:06.799687 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 2079
I20260809 23:12:06.799934 2411 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:12:06 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:55692
I20260809 23:12:06.809424 2799 tablet_replica.cc:330] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: stopping tablet replica
I20260809 23:12:06.809613 2799 raft_consensus.cc:2229] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:12:06.809779 2799 raft_consensus.cc:2258] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:12:06.812650 2799 ts_tablet_manager.cc:1905] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:12:06.814638 2798 tablet_replica.cc:330] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: stopping tablet replica
I20260809 23:12:06.814811 2798 raft_consensus.cc:2229] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:12:06.814949 2798 raft_consensus.cc:2258] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:12:06.818881 2799 ts_tablet_manager.cc:1918] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:12:06.818974 2799 log.cc:1196] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/6c8cc2d2efa6479b90d86abb26c8dcfd
I20260809 23:12:06.819267 2799 ts_tablet_manager.cc:1939] T 6c8cc2d2efa6479b90d86abb26c8dcfd P d1c6fb36dbb3458285da323cf8e442a8: Deleting consensus metadata
I20260809 23:12:06.819978 2798 ts_tablet_manager.cc:1905] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:12:06.826467 1949 catalog_manager.cc:4841] TS d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845): tablet 6c8cc2d2efa6479b90d86abb26c8dcfd (table test-table [id=8e9c079ee99240fc8ea061bb1c400fd1]) successfully deleted
I20260809 23:12:06.826598 2798 ts_tablet_manager.cc:1918] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:12:06.826713 2798 log.cc:1196] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/6c8cc2d2efa6479b90d86abb26c8dcfd
I20260809 23:12:06.827060 2798 ts_tablet_manager.cc:1939] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 1a500275092543c68b29f97e1878e2df: Deleting consensus metadata
I20260809 23:12:06.827610 1950 catalog_manager.cc:4841] TS 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669): tablet 6c8cc2d2efa6479b90d86abb26c8dcfd (table test-table [id=8e9c079ee99240fc8ea061bb1c400fd1]) successfully deleted
W20260809 23:12:06.830858 1951 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:37135 (error 108)
I20260809 23:12:06.831482 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.831310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37135 (local address 127.0.0.1:56290)
0809 23:12:06.831400 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.831449 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 23:12:06.831518 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 2308
W20260809 23:12:06.831684 1951 catalog_manager.cc:4584] TS 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193:37135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37135: connect: Connection refused (error 111)
I20260809 23:12:06.860687 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 2533
I20260809 23:12:06.885203 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.885042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37135 (local address 127.0.0.1:56292)
0809 23:12:06.885105 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.885145 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 23:12:06.885744 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:06.885620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37135 (local address 127.0.0.1:56298)
0809 23:12:06.885670 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:06.885701 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 23:12:06.914204 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 1918
I20260809 23:12:26.932921 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42437
--webserver_interface=127.8.141.254
--webserver_port=33267
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:12:27.027390 2801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:27.027621 2801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:27.027643 2801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:27.028784 2801 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:12:27.028820 2801 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:12:27.028834 2801 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:12:27.028846 2801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:12:27.028859 2801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:27.028872 2801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:27.028885 2801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:12:27.028898 2801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:12:27.030781 2801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=33267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:27.031279 2801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:27.031502 2801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:27.041538 2808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:27.041539 2814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:27.041538 2809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:27.041568 2801 server_base.cc:1034] running on GCE node
I20260809 23:12:27.042284 2801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:27.042654 2801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:27.043845 2801 hybrid_clock.cc:648] HybridClock initialized: now 1786317147043801 us; error 56 us; skew 500 ppm
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:27.047032 2801 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:12:27.047616 2801 webserver.cc:460] Webserver started at http://127.8.141.254:33267/ using document root <none> and password file <none>
I20260809 23:12:27.047842 2801 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:27.047904 2801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:27.049679 2801 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:12:27.050487 2822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.050729 2801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:27.050835 2801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "53f5878cb7f94ab0bde29da00fbc1c41"
format_stamp: "Formatted at 2026-08-09 23:12:05 on dist-test-slave-j2vl"
server_key: "3469eaeb40fb8d96c2a9dd9a7c4ac89f"
server_key_iv: "113f76df257fe94e7329a8553a45fb71"
server_key_version: "encryptionkey@0"
I20260809 23:12:27.051295 2801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:27.062311 2801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:27.065062 2801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:27.065609 2801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:27.073376 2801 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42437
I20260809 23:12:27.073464 2901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42437 every 8 connection(s)
I20260809 23:12:27.074136 2801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:12:27.075541 2902 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:12:27.076211 2902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Bootstrap starting.
I20260809 23:12:27.082834 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 2801
I20260809 23:12:27.083173 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:37135
--local_ip_for_outbound_sockets=127.8.141.193
--tserver_master_addrs=127.8.141.254:42437
--webserver_port=42889
--webserver_interface=127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:12:27.086390 2902 log.cc:824] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:27.089308 2902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:12:27.089763 2902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Bootstrap complete.
I20260809 23:12:27.092316 2902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:27.092597 2902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f5878cb7f94ab0bde29da00fbc1c41, State: Initialized, Role: FOLLOWER
I20260809 23:12:27.092819 2902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:27.092928 2902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:12:27.092993 2902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:12:27.093042 2902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 1 FOLLOWER]: Advancing to term 2
I20260809 23:12:27.094277 2902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:27.094415 2902 leader_election.cc:302] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f5878cb7f94ab0bde29da00fbc1c41; no voters:
I20260809 23:12:27.094615 2902 leader_election.cc:288] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 23:12:27.094691 2912 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 23:12:27.094880 2912 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 LEADER]: Becoming Leader. State: Replica: 53f5878cb7f94ab0bde29da00fbc1c41, State: Running, Role: LEADER
I20260809 23:12:27.094956 2902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:12:27.094983 2912 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:27.095248 2912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f5878cb7f94ab0bde29da00fbc1c41. Latest consensus state: current_term: 2 leader_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } } }
I20260809 23:12:27.095448 2912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: This master's current role is: LEADER
I20260809 23:12:27.095249 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } } }
I20260809 23:12:27.095760 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: This master's current role is: LEADER
I20260809 23:12:27.095968 2927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:12:27.096843 2927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:12:27.096949 2927 catalog_manager.cc:1259] Loaded cluster ID: eddc596b3d124ed8b36a874efee927b4
I20260809 23:12:27.096973 2927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:12:27.098304 2927 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:12:27.098520 2927 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Loaded TSK: 0
I20260809 23:12:27.114640 2927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Generated new TSK 1
I20260809 23:12:27.114800 2927 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:12:27.182802 2906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:27.183063 2906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:27.183094 2906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:27.184489 2906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:27.184546 2906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:27.184578 2906 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:12:27.186743 2906 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:37135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=42889
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2906
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:27.187083 2906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:27.187386 2906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:27.193190 2953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:27.193194 2959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:27.193562 2954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:27.193648 2906 server_base.cc:1034] running on GCE node
I20260809 23:12:27.194182 2906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:27.194550 2906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:27.195760 2906 hybrid_clock.cc:648] HybridClock initialized: now 1786317147195735 us; error 35 us; skew 500 ppm
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:27.199235 2906 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:12:27.199783 2906 webserver.cc:460] Webserver started at http://127.8.141.193:42889/ using document root <none> and password file <none>
I20260809 23:12:27.200035 2906 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:27.200129 2906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:27.201784 2906 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:12:27.202678 2968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.202886 2906 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:27.203024 2906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "76b6aaa363b64a55a30b1d9979c76e35"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "c61d1085ef597edf56459cb807143a49"
server_key_iv: "3812068746d66105d989d041e0a021f8"
server_key_version: "encryptionkey@0"
I20260809 23:12:27.203398 2906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:27.215068 2906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:27.217499 2906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:27.217669 2906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:27.217924 2906 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:27.218395 2981 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:12:27.220643 2906 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:12:27.220731 2906 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 23:12:27.220870 2906 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:12:27.221652 2906 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:12:27.221709 2906 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:27.221788 2981 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap starting.
I20260809 23:12:27.229477 2906 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:37135
I20260809 23:12:27.229664 3152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:37135 every 8 connection(s)
I20260809 23:12:27.229992 2906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:12:27.230126 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 2906
I20260809 23:12:27.230595 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:37845
--local_ip_for_outbound_sockets=127.8.141.194
--tserver_master_addrs=127.8.141.254:42437
--webserver_port=33853
--webserver_interface=127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:12:27.234448 2981 log.cc:824] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:27.237316 2981 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:12:27.237689 2981 tablet_bootstrap.cc:492] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap complete.
I20260809 23:12:27.239167 2981 ts_tablet_manager.cc:1397] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.015s
I20260809 23:12:27.241032 2981 raft_consensus.cc:348] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:27.241628 2981 raft_consensus.cc:729] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b6aaa363b64a55a30b1d9979c76e35, State: Initialized, Role: FOLLOWER
I20260809 23:12:27.241817 2981 consensus_queue.cc:260] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } }
I20260809 23:12:27.242125 2981 ts_tablet_manager.cc:1428] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Time spent starting tablet: real 0.003s user 0.002s sys 0.003s
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:12:27.251005 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.233664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:47315)
0809 23:12:27.234038 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.234089 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.234945 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.237003 (+ 2058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.237008 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.237485 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:27.237840 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.237849 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.239993 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.239995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.241479 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.241483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.242742 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.242743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.242887 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.243579 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:27.243589 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:27.245794 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:27.248834 (+ 3040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.248845 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.248847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.249165 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.249548 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.249549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.249550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.249707 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.250293 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:27.250301 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:27.250472 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.250582 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.250772 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":135,"threads_started":1}
I20260809 23:12:27.251048 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.234105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:47315 (local address 127.8.141.254:42437)
0809 23:12:27.236433 (+ 2328us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.236446 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.236500 (+ 54us) server_negotiation.cc:407] Connection header received
0809 23:12:27.236660 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.236668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.236749 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.238114 (+ 1365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:27.238223 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.239847 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.241800 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.242622 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.242771 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.245934 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:27.245940 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:27.245940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:27.245956 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:27.248703 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.249285 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.249289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.249289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.249467 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.249850 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.249852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.249852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.250196 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:27.250297 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.250682 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.250852 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2253,"thread_start_us":96,"threads_started":1}
I20260809 23:12:27.252866 3153 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42437
I20260809 23:12:27.253001 3153 heartbeater.cc:464] Registering TS with master...
I20260809 23:12:27.253206 3153 heartbeater.cc:511] Master 127.8.141.254:42437 requested a full tablet report, sending...
I20260809 23:12:27.254288 2852 ts_manager.cc:194] Registered new tserver with Master: 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193:37135)
I20260809 23:12:27.255599 2852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:47315
I20260809 23:12:27.256703 3153 heartbeater.cc:503] Master 127.8.141.254:42437 was elected leader, sending a full tablet report...
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.193@KRBTEST.COM
I20260809 23:12:27.265326 3168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.255008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37135 (local address 127.0.0.1:46522)
0809 23:12:27.255262 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.255295 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.255362 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.255592 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.255597 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.255950 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:27.256017 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.256020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.258164 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.258165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.258811 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.258814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.259665 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.259666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.259739 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.260315 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:27.260320 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:27.261838 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:27.263997 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.264006 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.264008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.264237 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.264529 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.264530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.264530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.264602 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.264978 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:27.264985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:27.265041 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.265121 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.265237 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":113,"threads_started":1}
I20260809 23:12:27.265451 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.255118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46522 (local address 127.8.141.193:37135)
0809 23:12:27.255327 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.255332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.255345 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:12:27.255465 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.255468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.255484 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.255567 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:27.256916 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.258059 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.258996 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.259519 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.259609 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.261932 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:27.261943 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:27.261943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:27.261952 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:27.263868 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.264341 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.264344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.264344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.264417 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.264677 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.264678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.264679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.264884 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:27.264975 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.265177 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.265290 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260809 23:12:27.265956 3044 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c8cc2d2efa6479b90d86abb26c8dcfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:12:06 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:46522
I20260809 23:12:27.266551 3170 tablet_replica.cc:330] stopping tablet replica
I20260809 23:12:27.266674 3170 raft_consensus.cc:2229] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:12:27.266752 3170 raft_consensus.cc:2258] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:12:27.267082 3170 ts_tablet_manager.cc:1905] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:12:27.269275 3170 ts_tablet_manager.cc:1918] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:12:27.269405 3170 log.cc:1196] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal/wals/6c8cc2d2efa6479b90d86abb26c8dcfd
I20260809 23:12:27.269654 3170 ts_tablet_manager.cc:1939] T 6c8cc2d2efa6479b90d86abb26c8dcfd P 76b6aaa363b64a55a30b1d9979c76e35: Deleting consensus metadata
I20260809 23:12:27.270269 2829 catalog_manager.cc:4841] TS 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193:37135): tablet 6c8cc2d2efa6479b90d86abb26c8dcfd (table test-table [id=8e9c079ee99240fc8ea061bb1c400fd1]) successfully deleted
W20260809 23:12:27.331622 3161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:27.331846 3161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:27.331872 3161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:27.333041 3161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:27.333083 3161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:27.333109 3161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:12:27.334607 3161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:37845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=33853
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:27.334841 3161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:27.335049 3161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:27.340654 3178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:27.340631 3181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:27.340943 3161 server_base.cc:1034] running on GCE node
W20260809 23:12:27.340631 3176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:27.341327 3161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:27.341665 3161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:27.342880 3161 hybrid_clock.cc:648] HybridClock initialized: now 1786317147342867 us; error 34 us; skew 500 ppm
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:27.345969 3161 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:12:27.346491 3161 webserver.cc:460] Webserver started at http://127.8.141.194:33853/ using document root <none> and password file <none>
I20260809 23:12:27.346783 3161 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:27.346897 3161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:27.348431 3161 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 23:12:27.349258 3192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.349485 3161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:27.349611 3161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "d1c6fb36dbb3458285da323cf8e442a8"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "752160e9907f530444e43806e511abdc"
server_key_iv: "d4034d275280c92121501c4e5bc892b9"
server_key_version: "encryptionkey@0"
I20260809 23:12:27.350028 3161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:27.368392 3161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:27.370703 3161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:27.370882 3161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:27.371142 3161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:27.371474 3161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:12:27.371531 3161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.371572 3161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:12:27.371614 3161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.377415 3161 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:37845
I20260809 23:12:27.377454 3362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:37845 every 8 connection(s)
I20260809 23:12:27.377825 3161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:12:27.378906 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 3161
I20260809 23:12:27.379179 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:39669
--local_ip_for_outbound_sockets=127.8.141.195
--tserver_master_addrs=127.8.141.254:42437
--webserver_port=46473
--webserver_interface=127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:12:27.388219 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.378826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:48699 (local address 127.8.141.254:42437)
0809 23:12:27.378887 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.378889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.378988 (+ 99us) server_negotiation.cc:407] Connection header received
0809 23:12:27.379669 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.379671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.379689 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.379773 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:27.380726 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.381380 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.382668 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.382902 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.382993 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.385235 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:27.385237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:27.385238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:27.385245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:27.386768 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.387194 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.387196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.387196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.387262 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.387531 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.387532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.387532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.387658 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:27.387736 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.388022 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.388124 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:12:27.388406 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.378678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:48699)
0809 23:12:27.378843 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.378885 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.379549 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.379823 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.379827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.380407 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:27.380611 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.380615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.381473 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.381474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.382571 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.382574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.383007 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.383008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.383147 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.383722 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:27.383731 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:27.385132 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:27.386867 (+ 1735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.386875 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.386877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.387116 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.387332 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.387333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.387333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.387454 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.387726 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:27.387733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:27.387880 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.387998 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.388163 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260809 23:12:27.388957 3363 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42437
I20260809 23:12:27.389057 3363 heartbeater.cc:464] Registering TS with master...
I20260809 23:12:27.389295 3363 heartbeater.cc:511] Master 127.8.141.254:42437 requested a full tablet report, sending...
I20260809 23:12:27.389775 2852 ts_manager.cc:194] Registered new tserver with Master: d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
I20260809 23:12:27.390161 2852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:48699
W20260809 23:12:27.466662 3368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:27.467027 3368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:27.467074 3368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:27.468770 3368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:27.468835 3368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:27.468868 3368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:12:27.470561 3368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:39669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=46473
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:27.470854 3368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:27.471138 3368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:27.476831 3379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:27.476831 3376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:27.476852 3374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:27.476958 3368 server_base.cc:1034] running on GCE node
I20260809 23:12:27.477408 3368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:27.477705 3368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:27.478891 3368 hybrid_clock.cc:648] HybridClock initialized: now 1786317147478881 us; error 71 us; skew 500 ppm
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:27.482172 3368 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:12:27.482717 3368 webserver.cc:460] Webserver started at http://127.8.141.195:46473/ using document root <none> and password file <none>
I20260809 23:12:27.482967 3368 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:27.483059 3368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:27.484649 3368 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:12:27.485453 3393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.485641 3368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:12:27.485749 3368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "1a500275092543c68b29f97e1878e2df"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "94a047cd718e64c042dece2ba9f7fc7b"
server_key_iv: "b0021f8e2c7a4f4ce6d2956f39ee9a56"
server_key_version: "encryptionkey@0"
I20260809 23:12:27.486164 3368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:27.500555 3368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:27.503393 3368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:27.503592 3368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:27.503890 3368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:27.504262 3368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:12:27.504324 3368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.504374 3368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:12:27.504427 3368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:27.511623 3368 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:39669
I20260809 23:12:27.511673 3585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:39669 every 8 connection(s)
I20260809 23:12:27.512097 3368 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:12:27.515800 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 3368
Aug 09 23:12:27 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:12:27.523274 3164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.512859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:47747 (local address 127.8.141.254:42437)
0809 23:12:27.512949 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.512952 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.513147 (+ 195us) server_negotiation.cc:407] Connection header received
0809 23:12:27.513804 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.513806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.513821 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.513877 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:27.514911 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.515418 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.517128 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.517397 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.517476 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.519757 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:27.519759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:27.519760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:27.519770 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:27.521600 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.522077 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.522079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.522079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.522144 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.522438 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.522439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.522440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.522601 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:27.522688 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.523037 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.523145 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:12:27.523463 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.512826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:47747)
0809 23:12:27.513003 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.513054 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.513690 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.513949 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.513953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.514585 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:27.514803 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.514807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.515521 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.515522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.517035 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.517039 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.517487 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.517488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.517641 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.518218 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:27.518227 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:27.519653 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:27.521710 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.521719 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.521722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.521986 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.522221 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:27.522221 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:27.522222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:27.522367 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:27.522685 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:27.522694 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:27.522871 (+ 177us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.523000 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.523182 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":78,"threads_started":1}
I20260809 23:12:27.524084 3586 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42437
I20260809 23:12:27.524231 3586 heartbeater.cc:464] Registering TS with master...
I20260809 23:12:27.524267 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.516268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.8.141.254:42437)
0809 23:12:27.516835 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.516836 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.516844 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:12:27.516865 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.516865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.516876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.516966 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:27.517696 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.518364 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.519151 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.519428 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.519512 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.521251 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:27.521253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:27.521253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:27.521260 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:27.522811 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.523206 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.523207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.523208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.523292 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.523664 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.523665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.523665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.523818 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:27.524016 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.524054 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.524138 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":60,"threads_started":1}
I20260809 23:12:27.524693 3586 heartbeater.cc:511] Master 127.8.141.254:42437 requested a full tablet report, sending...
I20260809 23:12:27.525182 2852 ts_manager.cc:194] Registered new tserver with Master: 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669)
I20260809 23:12:27.525734 2852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:47747
I20260809 23:12:27.536463 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:12:27.539563 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.536928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55968 (local address 127.8.141.254:42437)
0809 23:12:27.536998 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.537001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.537155 (+ 154us) server_negotiation.cc:407] Connection header received
0809 23:12:27.537221 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.537222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.537260 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.537412 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:12:27.537578 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.538122 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.538899 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.539108 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.539161 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.539314 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:12:27.539366 (+ 52us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 23:12:27.539507 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55968: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 23:12:27.539655 3590 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55968: token expired
W20260809 23:12:27.539882 2737 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.141.254:42437)
I20260809 23:12:27.545962 3590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.540152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.8.141.254:42437)
0809 23:12:27.540208 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.540209 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.540224 (+ 15us) server_negotiation.cc:407] Connection header received
0809 23:12:27.540243 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.540243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.540251 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.540318 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:27.540776 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.541213 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.541959 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.542136 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.542192 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.543354 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:27.543356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:27.543356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:27.543363 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:27.544755 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.545126 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.545127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.545127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.545186 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:27.545460 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:27.545462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:27.545462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:27.545582 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:27.545668 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.545786 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.545854 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:12:27.547371 2852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:12:27.549053 2852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:12:27.551723 3044 tablet_service.cc:1467] Processing CreateTablet for tablet 06e0ee1d7bde499ab8ddcab3579a02c1 (DEFAULT_TABLE table=test-table [id=3c9985d56bf24bcea8c426f2dc4c0101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:12:27.551965 3044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06e0ee1d7bde499ab8ddcab3579a02c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:12:27.554453 2981 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap starting.
I20260809 23:12:27.554842 2981 tablet_bootstrap.cc:654] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Neither blocks nor log segments found. Creating new log.
I20260809 23:12:27.556346 3168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.551230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:39669 (local address 127.0.0.1:55720)
0809 23:12:27.551328 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.551341 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.551381 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.551726 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.551728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.551730 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:27.551805 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.551807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.553715 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.553716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.555085 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.555103 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.556036 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.556037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.556123 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.556128 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.556194 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.556251 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":95}
I20260809 23:12:27.556362 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.551319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.8.141.195:39669)
0809 23:12:27.551509 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.551514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.551526 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:12:27.551575 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.551577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.551596 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.551918 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:27.552013 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.553584 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.555206 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.555934 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.556038 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.556105 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.556215 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.556261 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":114,"threads_started":1}
I20260809 23:12:27.556448 2981 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: No bootstrap required, opened a new log
I20260809 23:12:27.556461 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.551560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.8.141.194:37845)
0809 23:12:27.551789 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.551793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.551847 (+ 54us) server_negotiation.cc:407] Connection header received
0809 23:12:27.551897 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.551903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.551921 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.552038 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:27.552138 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.553591 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.555282 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.555873 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.555977 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.556049 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.556308 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.556360 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
I20260809 23:12:27.556492 3592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.551347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37845 (local address 127.0.0.1:46280)
0809 23:12:27.551584 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.551594 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.551640 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.552041 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.552042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.552045 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:27.552083 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.552084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.553768 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.553770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.555051 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.555056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.555980 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.555981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.556048 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.556075 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.556133 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.556189 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":62,"threads_started":1}
I20260809 23:12:27.556507 2981 ts_tablet_manager.cc:1397] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:12:27.556905 2981 raft_consensus.cc:348] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.557017 2981 raft_consensus.cc:374] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:12:27.557109 2981 raft_consensus.cc:729] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b6aaa363b64a55a30b1d9979c76e35, State: Initialized, Role: FOLLOWER
I20260809 23:12:27.557180 2981 consensus_queue.cc:260] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.557303 2981 ts_tablet_manager.cc:1428] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 23:12:27.557821 3261 tablet_service.cc:1467] Processing CreateTablet for tablet 06e0ee1d7bde499ab8ddcab3579a02c1 (DEFAULT_TABLE table=test-table [id=3c9985d56bf24bcea8c426f2dc4c0101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:12:27.557962 3479 tablet_service.cc:1467] Processing CreateTablet for tablet 06e0ee1d7bde499ab8ddcab3579a02c1 (DEFAULT_TABLE table=test-table [id=3c9985d56bf24bcea8c426f2dc4c0101]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:12:27.558219 3261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06e0ee1d7bde499ab8ddcab3579a02c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:12:27.558248 3479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06e0ee1d7bde499ab8ddcab3579a02c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:12:27.562124 3601 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Bootstrap starting.
I20260809 23:12:27.562202 3600 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Bootstrap starting.
I20260809 23:12:27.562845 3600 tablet_bootstrap.cc:654] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Neither blocks nor log segments found. Creating new log.
I20260809 23:12:27.562860 3601 tablet_bootstrap.cc:654] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Neither blocks nor log segments found. Creating new log.
I20260809 23:12:27.563387 3600 log.cc:824] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:27.563412 3601 log.cc:824] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:27.564294 3600 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: No bootstrap required, opened a new log
I20260809 23:12:27.564371 3600 ts_tablet_manager.cc:1397] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:12:27.564697 3601 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: No bootstrap required, opened a new log
I20260809 23:12:27.564774 3601 ts_tablet_manager.cc:1397] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:12:27.566113 3600 raft_consensus.cc:348] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.566303 3600 raft_consensus.cc:374] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:12:27.566355 3600 raft_consensus.cc:729] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a500275092543c68b29f97e1878e2df, State: Initialized, Role: FOLLOWER
I20260809 23:12:27.566485 3600 consensus_queue.cc:260] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.566288 3601 raft_consensus.cc:348] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.566637 3601 raft_consensus.cc:374] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:12:27.566725 3601 raft_consensus.cc:729] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c6fb36dbb3458285da323cf8e442a8, State: Initialized, Role: FOLLOWER
I20260809 23:12:27.566815 3600 ts_tablet_manager.cc:1428] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:12:27.566834 3586 heartbeater.cc:503] Master 127.8.141.254:42437 was elected leader, sending a full tablet report...
I20260809 23:12:27.566869 3601 consensus_queue.cc:260] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.567193 3601 ts_tablet_manager.cc:1428] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:12:27.567229 3363 heartbeater.cc:503] Master 127.8.141.254:42437 was elected leader, sending a full tablet report...
W20260809 23:12:27.628500 3365 tablet.cc:2369] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:12:27.698449 3607 raft_consensus.cc:482] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:12:27.698644 3607 raft_consensus.cc:504] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.699095 3607 leader_election.cc:288] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845), 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193:37135)
I20260809 23:12:27.702135 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.699463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:49605 (local address 127.8.141.193:37135)
0809 23:12:27.699567 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.699569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.699576 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:12:27.699601 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.699602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.699626 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.699706 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:27.699880 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.700415 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.701373 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.701799 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.701903 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.701987 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.702021 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.702062 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 23:12:27.702134 3588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.699321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37135 (local address 127.8.141.195:49605)
0809 23:12:27.699389 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.699402 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.699448 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.699713 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.699714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.699717 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:27.699800 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.699802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.700504 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.700505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.701250 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.701252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.701892 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.701893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.701970 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.701974 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.702018 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.702061 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 23:12:27.702214 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.699471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:58347 (local address 127.8.141.194:37845)
0809 23:12:27.699518 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.699520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.699677 (+ 157us) server_negotiation.cc:407] Connection header received
0809 23:12:27.699758 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.699759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.699772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.699850 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:12:27.700018 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.700480 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.701449 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.701834 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.701921 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.701930 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.702081 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.702122 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:12:27.702275 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.699327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37845 (local address 127.8.141.195:58347)
0809 23:12:27.699595 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.699604 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:27.699690 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:27.699855 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:27.699856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:27.699858 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:12:27.699941 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.699946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.700568 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.700569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.701310 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:27.701311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.701921 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.701922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:27.701997 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.702002 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:12:27.702075 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:12:27.702115 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":23,"thread_start_us":141,"threads_started":1}
I20260809 23:12:27.702557 3077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e0ee1d7bde499ab8ddcab3579a02c1" candidate_uuid: "1a500275092543c68b29f97e1878e2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b6aaa363b64a55a30b1d9979c76e35" is_pre_election: true
I20260809 23:12:27.702595 3291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e0ee1d7bde499ab8ddcab3579a02c1" candidate_uuid: "1a500275092543c68b29f97e1878e2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c6fb36dbb3458285da323cf8e442a8" is_pre_election: true
I20260809 23:12:27.702733 3291 raft_consensus.cc:2454] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a500275092543c68b29f97e1878e2df in term 0.
I20260809 23:12:27.702734 3077 raft_consensus.cc:2454] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a500275092543c68b29f97e1878e2df in term 0.
I20260809 23:12:27.702961 3395 leader_election.cc:302] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a500275092543c68b29f97e1878e2df, d1c6fb36dbb3458285da323cf8e442a8; no voters:
I20260809 23:12:27.703104 3607 raft_consensus.cc:2781] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:12:27.703164 3607 raft_consensus.cc:482] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:12:27.703190 3607 raft_consensus.cc:3037] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:12:27.703927 3607 raft_consensus.cc:504] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.704093 3607 leader_election.cc:288] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [CANDIDATE]: Term 1 election: Requested vote from peers d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845), 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193:37135)
I20260809 23:12:27.704291 3291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e0ee1d7bde499ab8ddcab3579a02c1" candidate_uuid: "1a500275092543c68b29f97e1878e2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1c6fb36dbb3458285da323cf8e442a8"
I20260809 23:12:27.704329 3077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06e0ee1d7bde499ab8ddcab3579a02c1" candidate_uuid: "1a500275092543c68b29f97e1878e2df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b6aaa363b64a55a30b1d9979c76e35"
I20260809 23:12:27.704398 3291 raft_consensus.cc:3037] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:12:27.704435 3077 raft_consensus.cc:3037] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:12:27.705137 3291 raft_consensus.cc:2454] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a500275092543c68b29f97e1878e2df in term 1.
I20260809 23:12:27.705253 3077 raft_consensus.cc:2454] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a500275092543c68b29f97e1878e2df in term 1.
I20260809 23:12:27.705313 3395 leader_election.cc:302] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a500275092543c68b29f97e1878e2df, d1c6fb36dbb3458285da323cf8e442a8; no voters:
I20260809 23:12:27.705433 3607 raft_consensus.cc:2781] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:12:27.705617 3607 raft_consensus.cc:686] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 1 LEADER]: Becoming Leader. State: Replica: 1a500275092543c68b29f97e1878e2df, State: Running, Role: LEADER
I20260809 23:12:27.705731 3607 consensus_queue.cc:237] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:27.706669 2852 catalog_manager.cc:5495] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a500275092543c68b29f97e1878e2df (127.8.141.195). New cstate: current_term: 1 leader_uuid: "1a500275092543c68b29f97e1878e2df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } health_report { overall_health: UNKNOWN } } }
W20260809 23:12:27.734877 3155 tablet.cc:2369] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:12:27.739892 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.737494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.8.141.195:39669)
0809 23:12:27.737575 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:27.737576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:27.737587 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:12:27.737655 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:27.737656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:27.737674 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:27.737758 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:12:27.737932 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.738396 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.739121 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:27.739322 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:27.739396 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:27.739528 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:12:27.739633 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:12:27.739697 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:27.739729 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 23:12:27.739746 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 23:12:27.741816 3291 raft_consensus.cc:1261] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Refusing update from remote peer 1a500275092543c68b29f97e1878e2df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:12:27.741816 3077 raft_consensus.cc:1261] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Refusing update from remote peer 1a500275092543c68b29f97e1878e2df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:12:27.742190 3607 consensus_queue.cc:1035] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:12:27.742354 3612 consensus_queue.cc:1035] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [LEADER]: Connected to new peer: Peer: permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:12:27.744424 3617 mvcc.cc:205] Tried to move back new op lower bound from 7316755037147488256 to 7316755037003464704. Current Snapshot: MvccSnapshot[applied={T|T < 7316755037147488256}]
I20260809 23:12:27.745048 3618 mvcc.cc:205] Tried to move back new op lower bound from 7316755037147488256 to 7316755037003464704. Current Snapshot: MvccSnapshot[applied={T|T < 7316755037147488256 or (T in {7316755037147488256})}]
I20260809 23:12:27.745726 3620 mvcc.cc:205] Tried to move back new op lower bound from 7316755037147488256 to 7316755037003464704. Current Snapshot: MvccSnapshot[applied={T|T < 7316755037147488256 or (T in {7316755037147488256})}]
I20260809 23:12:27.747344 2849 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:12:27.747465 2849 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:12:27.748325 2849 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Sending DeleteTablet for 3 replicas of tablet 06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:27.748695 3479 tablet_service.cc:1514] Processing DeleteTablet for tablet 06e0ee1d7bde499ab8ddcab3579a02c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:12:27 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:55720
I20260809 23:12:27.748667 3261 tablet_service.cc:1514] Processing DeleteTablet for tablet 06e0ee1d7bde499ab8ddcab3579a02c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:12:27 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:46280
I20260809 23:12:27.748927 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 2906
I20260809 23:12:27.748944 3637 tablet_replica.cc:330] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: stopping tablet replica
I20260809 23:12:27.749055 3637 raft_consensus.cc:2229] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:12:27.749164 3637 raft_consensus.cc:2258] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:12:27.755179 3637 ts_tablet_manager.cc:1905] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:12:27.757282 3637 ts_tablet_manager.cc:1918] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:12:27.757400 3637 log.cc:1196] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:27.757690 3637 ts_tablet_manager.cc:1939] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 1a500275092543c68b29f97e1878e2df: Deleting consensus metadata
I20260809 23:12:27.763725 3638 tablet_replica.cc:330] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: stopping tablet replica
I20260809 23:12:27.763931 3638 raft_consensus.cc:2229] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:12:27.764077 3638 raft_consensus.cc:2258] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:12:27.764462 2827 catalog_manager.cc:4841] TS 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669): tablet 06e0ee1d7bde499ab8ddcab3579a02c1 (table test-table [id=3c9985d56bf24bcea8c426f2dc4c0101]) successfully deleted
I20260809 23:12:27.764732 3638 ts_tablet_manager.cc:1905] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:12:27.766602 3638 ts_tablet_manager.cc:1918] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:12:27.766714 3638 log.cc:1196] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:27.767010 3638 ts_tablet_manager.cc:1939] T 06e0ee1d7bde499ab8ddcab3579a02c1 P d1c6fb36dbb3458285da323cf8e442a8: Deleting consensus metadata
I20260809 23:12:27.767541 2824 catalog_manager.cc:4841] TS d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845): tablet 06e0ee1d7bde499ab8ddcab3579a02c1 (table test-table [id=3c9985d56bf24bcea8c426f2dc4c0101]) successfully deleted
W20260809 23:12:27.771731 2829 connection.cc:535] client connection to 127.8.141.193:37135 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 23:12:27.771874 2829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:12:27.772390 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 3161
I20260809 23:12:27.772432 3592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:27.772258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:37135 (local address 127.0.0.1:46530)
0809 23:12:27.772338 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:27.772391 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 23:12:27.783360 2829 catalog_manager.cc:4584] TS 76b6aaa363b64a55a30b1d9979c76e35 (127.8.141.193:37135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06e0ee1d7bde499ab8ddcab3579a02c1: Network error: Client connection negotiation failed: client connection to 127.8.141.193:37135: connect: Connection refused (error 111)
I20260809 23:12:27.787789 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 3368
I20260809 23:12:27.806728 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 2801
I20260809 23:12:47.827595 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42437
--webserver_interface=127.8.141.254
--webserver_port=33267
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:12:47.919214 3640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:47.919471 3640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:47.919512 3640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:47.920831 3640 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 23:12:47.920902 3640 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 23:12:47.920925 3640 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 23:12:47.920953 3640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:12:47.920982 3640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:47.921011 3640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:47.921037 3640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:12:47.921064 3640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:12:47.922837 3640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=33267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:47.923158 3640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:47.923419 3640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:47.930184 3648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:47.930423 3647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:47.930469 3640 server_base.cc:1034] running on GCE node
W20260809 23:12:47.930565 3655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:47.930833 3640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:47.931167 3640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:47.932322 3640 hybrid_clock.cc:648] HybridClock initialized: now 1786317167932289 us; error 64 us; skew 500 ppm
Aug 09 23:12:47 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317167, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:47.940752 3640 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:12:47.941298 3640 webserver.cc:460] Webserver started at http://127.8.141.254:33267/ using document root <none> and password file <none>
I20260809 23:12:47.941620 3640 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:47.941700 3640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:47.943269 3640 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:12:47.944120 3666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:47.944546 3640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:47.944684 3640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "53f5878cb7f94ab0bde29da00fbc1c41"
format_stamp: "Formatted at 2026-08-09 23:12:05 on dist-test-slave-j2vl"
server_key: "3469eaeb40fb8d96c2a9dd9a7c4ac89f"
server_key_iv: "113f76df257fe94e7329a8553a45fb71"
server_key_version: "encryptionkey@0"
I20260809 23:12:47.945077 3640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:47.967715 3640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:47.970250 3640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:47.970427 3640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:47.975461 3640 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42437
I20260809 23:12:47.975603 3743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42437 every 8 connection(s)
I20260809 23:12:47.975965 3640 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:12:47.977632 3744 sys_catalog.cc:263] Verifying existing consensus state
I20260809 23:12:47.978281 3744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Bootstrap starting.
I20260809 23:12:47.985185 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 3640
I20260809 23:12:47.985589 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:37135
--local_ip_for_outbound_sockets=127.8.141.193
--tserver_master_addrs=127.8.141.254:42437
--webserver_port=42889
--webserver_interface=127.8.141.193
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:12:47.989643 3744 log.cc:824] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:47.992355 3744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:12:47.992729 3744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Bootstrap complete.
I20260809 23:12:47.995272 3744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:47.995529 3744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f5878cb7f94ab0bde29da00fbc1c41, State: Initialized, Role: FOLLOWER
I20260809 23:12:47.995898 3744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:47.996011 3744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:12:47.996070 3744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:12:47.996138 3744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 2 FOLLOWER]: Advancing to term 3
I20260809 23:12:47.997272 3744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:47.997423 3744 leader_election.cc:302] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f5878cb7f94ab0bde29da00fbc1c41; no voters:
I20260809 23:12:47.997668 3744 leader_election.cc:288] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 23:12:47.997710 3750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 23:12:47.997840 3750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [term 3 LEADER]: Becoming Leader. State: Replica: 53f5878cb7f94ab0bde29da00fbc1c41, State: Running, Role: LEADER
I20260809 23:12:47.997946 3750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } }
I20260809 23:12:47.997975 3744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:12:47.998267 3752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } } }
I20260809 23:12:47.998359 3752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: This master's current role is: LEADER
I20260809 23:12:47.998595 3753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f5878cb7f94ab0bde29da00fbc1c41. Latest consensus state: current_term: 3 leader_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f5878cb7f94ab0bde29da00fbc1c41" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42437 } } }
I20260809 23:12:47.998701 3753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41 [sys.catalog]: This master's current role is: LEADER
I20260809 23:12:47.998811 3756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:12:47.999843 3756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:12:48.000121 3756 catalog_manager.cc:1259] Loaded cluster ID: eddc596b3d124ed8b36a874efee927b4
I20260809 23:12:48.000207 3756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:12:48.001108 3756 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:12:48.001302 3756 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Loaded TSK: 0
I20260809 23:12:48.001403 3756 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 53f5878cb7f94ab0bde29da00fbc1c41: Loaded TSK: 1
F20260809 23:12:48.001652 3756 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8249cf79ad google::LogMessage::Fail() at ??:0
@ 0x7f8249cfa88a google::LogMessage::SendToLog() at ??:0
@ 0x7f8249cf74de google::LogMessage::Flush() at ??:0
@ 0x7f8249cf8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f824c464104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f824c461d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f8251c511cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f8251c5148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f8251c51594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f8251c17c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f8251c21ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f824bbc50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f824bbba954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f824ca1e6db start_thread at ??:0
@ 0x7f82487f561f clone at ??:0
W20260809 23:12:48.090723 3746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:48.090977 3746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:48.091018 3746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:48.092190 3746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:48.092257 3746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:48.092301 3746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:12:48.093909 3746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:37135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=42889
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:48.094180 3746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:48.094416 3746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:48.099702 3806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:48.099707 3809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:48.099720 3805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:48.100004 3746 server_base.cc:1034] running on GCE node
I20260809 23:12:48.100262 3746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:48.100543 3746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:48.101694 3746 hybrid_clock.cc:648] HybridClock initialized: now 1786317168101679 us; error 32 us; skew 500 ppm
Aug 09 23:12:48 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317168, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:48.104223 3746 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:12:48.104700 3746 webserver.cc:460] Webserver started at http://127.8.141.193:42889/ using document root <none> and password file <none>
I20260809 23:12:48.104947 3746 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:48.105018 3746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:48.106459 3746 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:48.107177 3822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:48.107328 3746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 23:12:48.107439 3746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "76b6aaa363b64a55a30b1d9979c76e35"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "c61d1085ef597edf56459cb807143a49"
server_key_iv: "3812068746d66105d989d041e0a021f8"
server_key_version: "encryptionkey@0"
I20260809 23:12:48.107806 3746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:48.121275 3746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:48.123829 3746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:48.123981 3746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:48.124250 3746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:48.124719 3830 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 23:12:48.126847 3746 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 23:12:48.126928 3746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 23:12:48.126988 3746 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 23:12:48.127671 3746 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 23:12:48.127729 3746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 23:12:48.127843 3830 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap starting.
I20260809 23:12:48.135608 3746 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:37135
I20260809 23:12:48.136040 3746 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:12:48.136619 3994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:37135 every 8 connection(s)
I20260809 23:12:48.140412 3830 log.cc:824] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Log is configured to *not* fsync() on all Append() calls
I20260809 23:12:48.143323 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 3746
I20260809 23:12:48.143420 3830 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 23:12:48.143654 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:37845
--local_ip_for_outbound_sockets=127.8.141.194
--tserver_master_addrs=127.8.141.254:42437
--webserver_port=33853
--webserver_interface=127.8.141.194
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:12:48.143755 3830 tablet_bootstrap.cc:492] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Bootstrap complete.
I20260809 23:12:48.145179 3830 ts_tablet_manager.cc:1397] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.004s
I20260809 23:12:48.146719 3830 raft_consensus.cc:348] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:48.147063 3830 raft_consensus.cc:729] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b6aaa363b64a55a30b1d9979c76e35, State: Initialized, Role: FOLLOWER
I20260809 23:12:48.147217 3830 consensus_queue.cc:260] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:48.147541 3830 ts_tablet_manager.cc:1428] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
W20260809 23:12:48.224941 3998 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:12:48.136897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:56133)
0809 23:12:48.137110 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.137151 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:48.137756 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:48.224723 (+ 86967us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":112,"threads_started":1}
W20260809 23:12:48.225324 3995 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42437 (0 consecutive failures): Network error: Failed to ping master at 127.8.141.254:42437: Client connection negotiation failed: client connection to 127.8.141.254:42437: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:12:48.225783 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.225633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:40561)
0809 23:12:48.225694 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.225753 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:12:48.226464 3998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.226327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:56553)
0809 23:12:48.226395 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.226440 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 23:12:48.226650 3995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 23:12:48.236570 4000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:48.236860 4000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:48.236908 4000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:48.238497 4000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:48.238606 4000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:48.238665 4000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:12:48.240617 4000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:37845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=33853
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:48.240960 4000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:48.241250 4000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:48.247370 4019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:48.247299 4016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:48.247329 4014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:48.247707 4000 server_base.cc:1034] running on GCE node
I20260809 23:12:48.247968 4000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:48.248240 4000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:48.249446 4000 hybrid_clock.cc:648] HybridClock initialized: now 1786317168249419 us; error 52 us; skew 500 ppm
Aug 09 23:12:48 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317168, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:48.252276 4000 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:12:48.252714 4000 webserver.cc:460] Webserver started at http://127.8.141.194:33853/ using document root <none> and password file <none>
I20260809 23:12:48.253008 4000 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:48.253086 4000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:48.254601 4000 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:48.255574 4028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:48.255842 4000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:48.255980 4000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "d1c6fb36dbb3458285da323cf8e442a8"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "752160e9907f530444e43806e511abdc"
server_key_iv: "d4034d275280c92121501c4e5bc892b9"
server_key_version: "encryptionkey@0"
I20260809 23:12:48.256373 4000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:48.268195 4000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:48.270697 4000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:48.270879 4000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:48.271140 4000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:48.271466 4000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:12:48.271523 4000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:48.271564 4000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:12:48.271595 4000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:48.277938 4000 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:37845
I20260809 23:12:48.277992 4219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:37845 every 8 connection(s)
I20260809 23:12:48.278350 4000 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:12:48.279502 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.279120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:48359)
0809 23:12:48.279273 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.279349 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
W20260809 23:12:48.279927 4223 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42437 (0 consecutive failures): Network error: Failed to ping master at 127.8.141.254:42437: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
I20260809 23:12:48.280220 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.280125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:59455)
0809 23:12:48.280174 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.280202 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 23:12:48.280521 4225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.280427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:58985)
0809 23:12:48.280473 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.280506 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 23:12:48.280679 4223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:12:48.280956 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 4000
I20260809 23:12:48.281246 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:39669
--local_ip_for_outbound_sockets=127.8.141.195
--tserver_master_addrs=127.8.141.254:42437
--webserver_port=46473
--webserver_interface=127.8.141.195
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:12:48.367781 4226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:12:48.367995 4226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:12:48.368021 4226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:12:48.369244 4226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:12:48.369290 4226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:12:48.369319 4226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:12:48.370854 4226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:39669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=46473
--tserver_master_addrs=127.8.141.254:42437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:12:48.371107 4226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:12:48.371376 4226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:12:48.376786 4234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:48.376758 4238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:12:48.376758 4235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:12:48.377260 4226 server_base.cc:1034] running on GCE node
I20260809 23:12:48.377559 4226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:12:48.377822 4226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:12:48.378996 4226 hybrid_clock.cc:648] HybridClock initialized: now 1786317168378984 us; error 33 us; skew 500 ppm
Aug 09 23:12:48 dist-test-slave-j2vl krb5kdc[1898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317168, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:12:48.381816 4226 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:12:48.382381 4226 webserver.cc:460] Webserver started at http://127.8.141.195:46473/ using document root <none> and password file <none>
I20260809 23:12:48.382624 4226 fs_manager.cc:359] Metadata directory not provided
I20260809 23:12:48.382699 4226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:12:48.384341 4226 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:12:48.385161 4248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:48.385562 4226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:12:48.385687 4226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "1a500275092543c68b29f97e1878e2df"
format_stamp: "Formatted at 2026-08-09 23:12:06 on dist-test-slave-j2vl"
server_key: "94a047cd718e64c042dece2ba9f7fc7b"
server_key_iv: "b0021f8e2c7a4f4ce6d2956f39ee9a56"
server_key_version: "encryptionkey@0"
I20260809 23:12:48.386116 4226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:12:48.404744 4226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:12:48.407816 4226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:12:48.408015 4226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:12:48.408291 4226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:12:48.408646 4226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:12:48.408705 4226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:48.408751 4226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:12:48.408778 4226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:12:48.416172 4226 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:39669
I20260809 23:12:48.416234 4442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:39669 every 8 connection(s)
I20260809 23:12:48.416617 4226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:12:48.417546 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 4226
I20260809 23:12:48.418212 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.417663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:60273)
0809 23:12:48.417903 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.417981 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":123,"threads_started":1}
W20260809 23:12:48.418810 4443 heartbeater.cc:644] Failed to heartbeat to 127.8.141.254:42437 (0 consecutive failures): Network error: Failed to ping master at 127.8.141.254:42437: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
I20260809 23:12:48.419209 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.419067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:44653)
0809 23:12:48.419151 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.419183 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 23:12:48.419720 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:48.419574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:58811)
0809 23:12:48.419664 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:48.419696 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 23:12:48.420069 4443 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 23:12:49.227406 4449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:49.227107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:59713)
0809 23:12:49.227265 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:49.227349 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":69,"threads_started":1}
I20260809 23:12:49.281422 4450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:49.281113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:51391)
0809 23:12:49.281252 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:49.281295 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260809 23:12:49.420799 4451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:49.420528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:41959)
0809 23:12:49.420674 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:49.420718 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
I20260809 23:12:49.871515 4453 raft_consensus.cc:482] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:12:49.871658 4453 raft_consensus.cc:504] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:49.872035 4453 leader_election.cc:288] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669), d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
Aug 09 23:12:49 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317168, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.194@KRBTEST.COM
Aug 09 23:12:49 dist-test-slave-j2vl krb5kdc[1898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317168, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.195@KRBTEST.COM
I20260809 23:12:49.885946 4457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:49.872414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:43923 (local address 127.8.141.194:37845)
0809 23:12:49.872618 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:49.872631 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:49.872656 (+ 25us) server_negotiation.cc:407] Connection header received
0809 23:12:49.873257 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:49.873261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:49.873377 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:49.873544 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:49.874767 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.876237 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.877940 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.878609 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.878739 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:49.881743 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:49.881749 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:49.881749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:49.881763 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:49.883902 (+ 2139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:49.884509 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:49.884513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:49.884513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:49.884636 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:49.884927 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:49.884928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:49.884928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:49.885240 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:49.885304 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:49.885695 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 23:12:49.885828 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":63,"threads_started":1}
I20260809 23:12:49.885949 4459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:49.872446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:42399 (local address 127.8.141.195:39669)
0809 23:12:49.872618 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 23:12:49.872631 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:12:49.872668 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:12:49.873262 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:12:49.873264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:12:49.873377 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:12:49.874074 (+ 697us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:12:49.874790 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.876237 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.877940 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.878609 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.878739 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:49.882115 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:12:49.882121 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:12:49.882121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:12:49.882137 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:12:49.884208 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:49.884715 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:49.884718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:49.884718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:49.884826 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:12:49.885068 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:12:49.885069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:12:49.885069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:12:49.885344 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:12:49.885462 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 23:12:49.885672 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 23:12:49.885809 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 23:12:49.885931 4454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:49.872259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:39669 (local address 127.8.141.193:42399)
0809 23:12:49.872418 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:49.872433 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:49.872503 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:49.873620 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:49.873624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:49.874260 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:49.874585 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:49.874593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.876459 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.876461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:49.877818 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:49.877821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.878734 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.878735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:49.878899 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:49.879656 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:49.879659 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:49.881993 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:49.884344 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:49.884345 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:49.884346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:49.884604 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:49.884925 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:49.884926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:49.884926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:49.884997 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:49.885427 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:49.885428 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:49.885500 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 23:12:49.885588 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 23:12:49.885820 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":59,"thread_start_us":77,"threads_started":1}
W20260809 23:12:49.886778 3826 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:49.886904 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:49.872276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37845 (local address 127.8.141.193:43923)
0809 23:12:49.872447 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:49.872460 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:12:49.872523 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:12:49.873622 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:12:49.873625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:12:49.874260 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:12:49.874585 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:49.874593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.876472 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.876473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:49.877818 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:12:49.877821 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:12:49.878735 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:12:49.878736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:12:49.878899 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:12:49.879624 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:12:49.879636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:12:49.881609 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:12:49.884042 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:49.884054 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:49.884057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:49.884406 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:49.884711 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:12:49.884711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:12:49.884712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:12:49.884856 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:12:49.885316 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:12:49.885324 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:12:49.885498 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 23:12:49.885614 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 23:12:49.885907 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":277,"thread_start_us":83,"threads_started":1}
W20260809 23:12:49.887594 3825 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:49.887648 3825 leader_election.cc:302] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76b6aaa363b64a55a30b1d9979c76e35; no voters: 1a500275092543c68b29f97e1878e2df, d1c6fb36dbb3458285da323cf8e442a8
I20260809 23:12:49.887759 4453 raft_consensus.cc:2726] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:12:50.228154 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:50.227982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:34437)
0809 23:12:50.228062 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:50.228112 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 23:12:50.282246 4460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:50.281973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:50465)
0809 23:12:50.282121 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:50.282174 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260809 23:12:50.421666 4461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:50.421427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:41461)
0809 23:12:50.421576 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:50.421619 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260809 23:12:51.229005 4464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:51.228741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:60327)
0809 23:12:51.228897 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:51.228945 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260809 23:12:51.283206 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:51.282949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:36749)
0809 23:12:51.283112 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:51.283156 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260809 23:12:51.422456 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:51.422214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:56619)
0809 23:12:51.422369 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:51.422410 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260809 23:12:51.655121 4468 raft_consensus.cc:482] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:12:51.655242 4468 raft_consensus.cc:504] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:51.655472 4468 leader_election.cc:288] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669), d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
W20260809 23:12:51.656102 3826 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
W20260809 23:12:51.656184 3825 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:51.656225 3825 leader_election.cc:302] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76b6aaa363b64a55a30b1d9979c76e35; no voters: 1a500275092543c68b29f97e1878e2df, d1c6fb36dbb3458285da323cf8e442a8
I20260809 23:12:51.656308 4468 raft_consensus.cc:2726] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:12:52.229980 4469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:52.229684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:33209)
0809 23:12:52.229836 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:52.229912 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260809 23:12:52.284054 4471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:52.283739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:52011)
0809 23:12:52.283948 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:52.283987 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":137,"threads_started":1}
I20260809 23:12:52.423409 4472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:52.423153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:44985)
0809 23:12:52.423313 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:52.423361 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
I20260809 23:12:53.230927 4473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:53.230669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:55283)
0809 23:12:53.230827 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:53.230874 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260809 23:12:53.284929 4474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:53.284631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:43113)
0809 23:12:53.284796 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:53.284876 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":70,"threads_started":1}
I20260809 23:12:53.426692 4475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:53.424750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:54213)
0809 23:12:53.426582 (+ 1832us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:53.426640 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1806,"thread_start_us":1775,"threads_started":1}
I20260809 23:12:53.964856 4476 raft_consensus.cc:482] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:12:53.964988 4476 raft_consensus.cc:504] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:53.965258 4476 leader_election.cc:288] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669), d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
W20260809 23:12:53.965866 3826 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
W20260809 23:12:53.966006 3825 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:53.966084 3825 leader_election.cc:302] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76b6aaa363b64a55a30b1d9979c76e35; no voters: 1a500275092543c68b29f97e1878e2df, d1c6fb36dbb3458285da323cf8e442a8
I20260809 23:12:53.966166 4476 raft_consensus.cc:2726] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:12:54.231848 4477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:54.231537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:41983)
0809 23:12:54.231747 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:54.231798 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260809 23:12:54.285792 4481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:54.285564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:34855)
0809 23:12:54.285693 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:54.285742 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260809 23:12:54.427683 4482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:54.427347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:51795)
0809 23:12:54.427581 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:54.427630 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":114,"threads_started":1}
I20260809 23:12:55.232867 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:55.232580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:54545)
0809 23:12:55.232763 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:55.232814 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":80,"threads_started":1}
I20260809 23:12:55.286695 4485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:55.286414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:55031)
0809 23:12:55.286591 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:55.286642 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260809 23:12:55.428701 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:55.428391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:43139)
0809 23:12:55.428602 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:55.428654 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":105,"threads_started":1}
I20260809 23:12:56.233775 4489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:56.233512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:57291)
0809 23:12:56.233663 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:56.233709 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260809 23:12:56.287467 4490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:56.287232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:34861)
0809 23:12:56.287376 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:56.287414 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260809 23:12:56.429786 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:56.429443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:39039)
0809 23:12:56.429677 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:56.429723 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":148,"threads_started":1}
I20260809 23:12:56.977571 4492 raft_consensus.cc:482] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:12:56.977698 4492 raft_consensus.cc:504] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:12:56.977892 4492 leader_election.cc:288] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669), d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
W20260809 23:12:56.978363 3825 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
W20260809 23:12:56.978443 3826 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:12:56.978492 3826 leader_election.cc:302] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76b6aaa363b64a55a30b1d9979c76e35; no voters: 1a500275092543c68b29f97e1878e2df, d1c6fb36dbb3458285da323cf8e442a8
I20260809 23:12:56.978566 4492 raft_consensus.cc:2726] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:12:57.234834 4495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:57.234479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:42773)
0809 23:12:57.234735 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:57.234782 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":79,"threads_started":1}
I20260809 23:12:57.288410 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:57.288138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:53337)
0809 23:12:57.288293 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:57.288341 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260809 23:12:57.430599 4497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:57.430360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:56099)
0809 23:12:57.430516 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:57.430551 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260809 23:12:58.235936 4498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:58.235631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:38841)
0809 23:12:58.235809 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:58.235860 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":74,"threads_started":1}
I20260809 23:12:58.289461 4501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:58.289053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:47127)
0809 23:12:58.289280 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:58.289382 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":92,"threads_started":1}
I20260809 23:12:58.431442 4503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:58.431161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:60761)
0809 23:12:58.431334 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:58.431391 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260809 23:12:59.236851 4504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:59.236580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:34537)
0809 23:12:59.236750 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:59.236800 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260809 23:12:59.290529 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:59.290181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:42337)
0809 23:12:59.290403 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:59.290475 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":85,"threads_started":1}
I20260809 23:12:59.432348 4506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:12:59.432083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:56091)
0809 23:12:59.432237 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 23:12:59.432296 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260809 23:13:00.237739 4510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:00.237469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:38145)
0809 23:13:00.237643 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:00.237687 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260809 23:13:00.291278 4512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:00.291027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:59071)
0809 23:13:00.291191 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:00.291229 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260809 23:13:00.433244 4513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:00.432989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:39383)
0809 23:13:00.433131 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:00.433178 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260809 23:13:01.238658 4514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:01.238388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:38261)
0809 23:13:01.238559 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:01.238608 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260809 23:13:01.292203 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:01.291905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:52387)
0809 23:13:01.292097 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:01.292153 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260809 23:13:01.434106 4523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:01.433825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:46267)
0809 23:13:01.433984 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:01.434019 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260809 23:13:02.239614 4526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:02.239336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:41331)
0809 23:13:02.239516 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:02.239563 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":73,"threads_started":1}
I20260809 23:13:02.293203 4527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:02.292903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:33809)
0809 23:13:02.293100 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:02.293148 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":102,"threads_started":1}
I20260809 23:13:02.434989 4530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:02.434694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:48767)
0809 23:13:02.434890 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:02.434939 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":79,"threads_started":1}
I20260809 23:13:02.446753 4531 raft_consensus.cc:482] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:13:02.446939 4531 raft_consensus.cc:504] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a500275092543c68b29f97e1878e2df" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 39669 } } peers { permanent_uuid: "d1c6fb36dbb3458285da323cf8e442a8" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37845 } } peers { permanent_uuid: "76b6aaa363b64a55a30b1d9979c76e35" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 37135 } }
I20260809 23:13:02.447155 4531 leader_election.cc:288] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669), d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845)
W20260809 23:13:02.447597 3826 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1a500275092543c68b29f97e1878e2df (127.8.141.195:39669): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
W20260809 23:13:02.447703 3825 leader_election.cc:341] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1c6fb36dbb3458285da323cf8e442a8 (127.8.141.194:37845): Not found: Tablet not found: 06e0ee1d7bde499ab8ddcab3579a02c1
I20260809 23:13:02.447781 3825 leader_election.cc:302] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76b6aaa363b64a55a30b1d9979c76e35; no voters: 1a500275092543c68b29f97e1878e2df, d1c6fb36dbb3458285da323cf8e442a8
I20260809 23:13:02.447888 4531 raft_consensus.cc:2726] T 06e0ee1d7bde499ab8ddcab3579a02c1 P 76b6aaa363b64a55a30b1d9979c76e35 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 23:13:03.240554 4533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:03.240330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.193:53697)
0809 23:13:03.240464 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:03.240507 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260809 23:13:03.294260 4534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:03.294015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.194:37305)
0809 23:13:03.294173 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:03.294212 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260809 23:13:03.418269 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 3746
I20260809 23:13:03.451740 4536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:03.451509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42437 (local address 127.8.141.195:57323)
0809 23:13:03.451644 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:03.451689 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.254:42437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260809 23:13:03.456943 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 4000
I20260809 23:13:03.472774 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 4226
2026-08-09T23:13:03Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59086 ms)
[----------] 30 tests from SecurityITest (257324 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:03 dist-test-slave-j2vl krb5kdc[4542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:03 dist-test-slave-j2vl krb5kdc[4542](info): set up 2 sockets
Aug 09 23:13:03 dist-test-slave-j2vl krb5kdc[4542](info): commencing operation
krb5kdc: starting...
W20260809 23:13:04.816287 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:04 dist-test-slave-j2vl krb5kdc[4542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:04.958282 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:37861
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:37861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:05.050020 4560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:05.050323 4560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:05.050369 4560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:05.052363 4560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:05.052670 4560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:05.052772 4560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:05.052809 4560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:05.052838 4560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:05.055171 4560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:37861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:37861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:05.055536 4560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:05.055811 4560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:05.061837 4568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.061992 4572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.061800 4569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:05.062449 4560 server_base.cc:1034] running on GCE node
I20260809 23:13:05.062767 4560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:05.063308 4560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:05.064577 4560 hybrid_clock.cc:648] HybridClock initialized: now 1786317185064522 us; error 65 us; skew 500 ppm
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:05.067642 4560 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:05.068176 4560 webserver.cc:460] Webserver started at http://127.8.141.254:46329/ using document root <none> and password file <none>
I20260809 23:13:05.068421 4560 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:05.068475 4560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:05.068620 4560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:05.069886 4560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "b7acfdb2dde0490a9e3d2113060ca5f0"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "457d51bd98f509009dabdd5eabc3dad8"
server_key_iv: "174ab74cbebae5c2ca7394540692ae8e"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.070266 4560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "b7acfdb2dde0490a9e3d2113060ca5f0"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "457d51bd98f509009dabdd5eabc3dad8"
server_key_iv: "174ab74cbebae5c2ca7394540692ae8e"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.071898 4560 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:05.072808 4583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.073133 4560 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:05.073240 4560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "b7acfdb2dde0490a9e3d2113060ca5f0"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "457d51bd98f509009dabdd5eabc3dad8"
server_key_iv: "174ab74cbebae5c2ca7394540692ae8e"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.073400 4560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:05.084829 4560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:05.087495 4560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:05.087673 4560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:05.092339 4560 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:37861
I20260809 23:13:05.092379 4671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:37861 every 8 connection(s)
I20260809 23:13:05.092760 4560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:05.093840 4672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:05.095209 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 4560
I20260809 23:13:05.095310 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:05.095533 8759 external_mini_cluster.cc:1428] Setting key 6f577b97b2df232ab781f77481e9f0f2
I20260809 23:13:05.096777 4672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0: Bootstrap starting.
I20260809 23:13:05.097581 4672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:05.097962 4672 log.cc:824] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:05.098759 4672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0: No bootstrap required, opened a new log
I20260809 23:13:05.101300 4672 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37861 } }
I20260809 23:13:05.101634 4672 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:05.101691 4672 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7acfdb2dde0490a9e3d2113060ca5f0, State: Initialized, Role: FOLLOWER
I20260809 23:13:05.101861 4672 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37861 } }
I20260809 23:13:05.101969 4672 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:05.102011 4672 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:05.102056 4672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:05.103004 4672 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37861 } }
I20260809 23:13:05.103108 4672 leader_election.cc:302] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7acfdb2dde0490a9e3d2113060ca5f0; no voters:
I20260809 23:13:05.103360 4672 leader_election.cc:288] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:05.103403 4682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 1 FOLLOWER]: Leader election won for term 1
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:05.103518 4682 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [term 1 LEADER]: Becoming Leader. State: Replica: b7acfdb2dde0490a9e3d2113060ca5f0, State: Running, Role: LEADER
I20260809 23:13:05.103598 4682 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37861 } }
I20260809 23:13:05.103662 4672 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:05.104036 4683 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37861 } } }
I20260809 23:13:05.104149 4683 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:05.104034 4684 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7acfdb2dde0490a9e3d2113060ca5f0. Latest consensus state: current_term: 1 leader_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7acfdb2dde0490a9e3d2113060ca5f0" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37861 } } }
I20260809 23:13:05.104542 4684 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:05.104895 4687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:05.105782 4687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:05.107590 4687 catalog_manager.cc:1347] Generated new cluster ID: 519d2e05b03b4518a67367f430a91f32
I20260809 23:13:05.107645 4687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:05.108817 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.096125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.8.141.254:37861)
0809 23:13:05.096322 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.096331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.096358 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:13:05.097008 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.097017 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.097072 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.097219 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:05.098139 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.099171 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.100316 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.100782 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.100872 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.104155 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:05.104160 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:05.104160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:05.104176 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:05.106477 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.106958 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.106961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.106961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.107125 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.107409 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.107411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.107411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.107755 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:05.108368 (+ 613us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.108454 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.108597 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260809 23:13:05.130178 4687 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:05.130990 4687 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:05.142782 4687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0: Generated new TSK 0
I20260809 23:13:05.143033 4687 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:13:05.186331 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37861
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:05.305078 4716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:05.305318 4716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:05.305397 4716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:05.306695 4716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:05.306746 4716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:05.306777 4716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:13:05.308387 4716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:05.308638 4716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:05.308933 4716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:05.314358 4727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.314363 4730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.314410 4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:05.314805 4716 server_base.cc:1034] running on GCE node
I20260809 23:13:05.315172 4716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:05.315490 4716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:05.316694 4716 hybrid_clock.cc:648] HybridClock initialized: now 1786317185316677 us; error 28 us; skew 500 ppm
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:05.319636 4716 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:13:05.320163 4716 webserver.cc:460] Webserver started at http://127.8.141.193:41935/ using document root <none> and password file <none>
I20260809 23:13:05.320413 4716 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:05.320484 4716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:05.320639 4716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:05.321808 4716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "a356b9120a994f5eb00b2f0141a91b7f"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "fbfd1deeea06e04729bf945ddde3fce3"
server_key_iv: "e05573245b2d217af0ce3a445c5e910a"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.322184 4716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "a356b9120a994f5eb00b2f0141a91b7f"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "fbfd1deeea06e04729bf945ddde3fce3"
server_key_iv: "e05573245b2d217af0ce3a445c5e910a"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.323794 4716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:05.324703 4745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.324939 4716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:05.325059 4716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "a356b9120a994f5eb00b2f0141a91b7f"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "fbfd1deeea06e04729bf945ddde3fce3"
server_key_iv: "e05573245b2d217af0ce3a445c5e910a"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.325142 4716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:05.363886 4716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:05.366456 4716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:05.366641 4716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:05.366914 4716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:05.367259 4716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:05.367314 4716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.367357 4716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:05.367403 4716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.374069 4716 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:44023
I20260809 23:13:05.374117 4925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:44023 every 8 connection(s)
I20260809 23:13:05.374462 4716 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:13:05.374662 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 4716
I20260809 23:13:05.374784 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:13:05.375066 8759 external_mini_cluster.cc:1428] Setting key d1d737c4c02cca6d0395be77f7c9d6c9
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
I20260809 23:13:05.385735 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.375275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:60313 (local address 127.8.141.254:37861)
0809 23:13:05.375336 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.375338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.375657 (+ 319us) server_negotiation.cc:407] Connection header received
0809 23:13:05.376436 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.376437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.376449 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.376508 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:05.377919 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.378720 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.379749 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.380001 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.380089 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.382642 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:05.382644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:05.382644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:05.382653 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:05.384211 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.384677 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.384678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.384679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.384740 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.384994 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.384996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.384996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.385119 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:05.385195 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.385546 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.385635 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:13:05.385960 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.375259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37861 (local address 127.8.141.193:60313)
0809 23:13:05.375500 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.375557 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.376324 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.376590 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.376594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.377507 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:05.377803 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.377810 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.378852 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.378853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.379662 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.379664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.380099 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.380101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.380228 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.380811 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:05.380820 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:05.382522 (+ 1702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:05.384340 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:05.384348 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:05.384351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:05.384585 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:05.384811 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:05.384811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:05.384812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:05.384925 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:05.385198 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:05.385206 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:05.385397 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.385526 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.385698 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":103,"threads_started":1}
I20260809 23:13:05.386551 4926 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37861
I20260809 23:13:05.386680 4926 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:05.386876 4926 heartbeater.cc:511] Master 127.8.141.254:37861 requested a full tablet report, sending...
I20260809 23:13:05.387442 4609 ts_manager.cc:194] Registered new tserver with Master: a356b9120a994f5eb00b2f0141a91b7f (127.8.141.193:44023)
I20260809 23:13:05.388151 4609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:60313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:13:05.409137 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37861
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:05.495272 4933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:05.495496 4933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:05.495522 4933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:05.496649 4933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:05.496687 4933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:05.496714 4933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:13:05.498167 4933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:05.498368 4933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:05.498550 4933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:05.504196 4944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.504196 4942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.504300 4941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:05.504670 4933 server_base.cc:1034] running on GCE node
I20260809 23:13:05.504871 4933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:05.505147 4933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:05.506297 4933 hybrid_clock.cc:648] HybridClock initialized: now 1786317185506274 us; error 39 us; skew 500 ppm
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:05.509186 4933 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:13:05.509752 4933 webserver.cc:460] Webserver started at http://127.8.141.194:35335/ using document root <none> and password file <none>
I20260809 23:13:05.510015 4933 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:05.510089 4933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:05.510255 4933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:05.511360 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "d8d2817b48dc4791937334311b8c87d4"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "65a8b834ee6a5321a577f1f584bb131a"
server_key_iv: "3e7aa189949b83ab6865ed80239fb038"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.511756 4933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "d8d2817b48dc4791937334311b8c87d4"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "65a8b834ee6a5321a577f1f584bb131a"
server_key_iv: "3e7aa189949b83ab6865ed80239fb038"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.513252 4933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:05.514262 4960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.514518 4933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:05.514635 4933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "d8d2817b48dc4791937334311b8c87d4"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "65a8b834ee6a5321a577f1f584bb131a"
server_key_iv: "3e7aa189949b83ab6865ed80239fb038"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.514742 4933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:05.525684 4933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:05.528303 4933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:05.528483 4933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:05.528745 4933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:05.529084 4933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:05.529184 4933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.529234 4933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:05.529251 4933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.535859 4933 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:41777
I20260809 23:13:05.535918 5151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:41777 every 8 connection(s)
I20260809 23:13:05.536250 4933 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:05.545953 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 4933
I20260809 23:13:05.546061 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:13:05.546482 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.537138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:58705 (local address 127.8.141.254:37861)
0809 23:13:05.537227 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.537230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.537383 (+ 153us) server_negotiation.cc:407] Connection header received
0809 23:13:05.537929 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.537930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.537943 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.538002 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:05.538988 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.539500 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.540560 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.540803 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.540873 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.543407 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:05.543409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:05.543409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:05.543417 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:05.544796 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.545244 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.545246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.545246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.545348 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.545684 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.545685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.545686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.545825 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:05.545916 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.546275 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.546374 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 23:13:05.546499 8759 external_mini_cluster.cc:1428] Setting key 4f82921ec440790b8f5ddbdfae913930
I20260809 23:13:05.546854 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.537026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37861 (local address 127.8.141.194:58705)
0809 23:13:05.537217 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.537266 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.537819 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.538057 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.538060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.538660 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:05.538862 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.538867 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.539609 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.539610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.540446 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.540449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.540931 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.540932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.541052 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.541663 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:05.541673 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:05.543285 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:05.544913 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:05.544920 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:05.544923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:05.545158 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:05.545468 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:05.545469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:05.545470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:05.545602 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:05.545922 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:05.545930 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:05.546122 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.546248 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.546459 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260809 23:13:05.547711 5152 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37861
I20260809 23:13:05.547844 5152 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:05.548089 5152 heartbeater.cc:511] Master 127.8.141.254:37861 requested a full tablet report, sending...
I20260809 23:13:05.548596 4607 ts_manager.cc:194] Registered new tserver with Master: d8d2817b48dc4791937334311b8c87d4 (127.8.141.194:41777)
I20260809 23:13:05.549067 4607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:58705
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:13:05.576928 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37861
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:45313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:05.671308 5159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:05.671548 5159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:05.671576 5159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:05.672837 5159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:05.672889 5159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:05.672919 5159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:13:05.674571 5159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:45313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:37861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:05.674868 5159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:05.675139 5159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:05.680975 5173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.680984 5176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:05.680991 5178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:05.681386 5159 server_base.cc:1034] running on GCE node
I20260809 23:13:05.681648 5159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:05.681943 5159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:05.683109 5159 hybrid_clock.cc:648] HybridClock initialized: now 1786317185683086 us; error 43 us; skew 500 ppm
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:05.686213 5159 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:13:05.686745 5159 webserver.cc:460] Webserver started at http://127.8.141.195:41987/ using document root <none> and password file <none>
I20260809 23:13:05.687044 5159 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:05.687096 5159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:05.687235 5159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:05.688385 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "09d8ffe9364240c8a57c9b6708eeef19"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "a8ed6d4000667f45eecec958f9d68cdc"
server_key_iv: "0083eb599252a8a51db7f314bde954e8"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.688768 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "09d8ffe9364240c8a57c9b6708eeef19"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "a8ed6d4000667f45eecec958f9d68cdc"
server_key_iv: "0083eb599252a8a51db7f314bde954e8"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.690397 5159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:05.691483 5187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.691740 5159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:05.691864 5159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "09d8ffe9364240c8a57c9b6708eeef19"
format_stamp: "Formatted at 2026-08-09 23:13:05 on dist-test-slave-j2vl"
server_key: "a8ed6d4000667f45eecec958f9d68cdc"
server_key_iv: "0083eb599252a8a51db7f314bde954e8"
server_key_version: "encryptionkey@0"
I20260809 23:13:05.691995 5159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:05.710367 5159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:05.713080 5159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:05.713294 5159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:05.713606 5159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:05.713959 5159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:05.714017 5159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.714066 5159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:05.714097 5159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:05.722093 5159 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:42029
I20260809 23:13:05.722142 5378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:42029 every 8 connection(s)
I20260809 23:13:05.722613 5159 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:13:05.723687 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 5159
I20260809 23:13:05.723796 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:13:05.724041 8759 external_mini_cluster.cc:1428] Setting key 82c7476a2a4c556fc4e4e372d3fca6f6
Aug 09 23:13:05 dist-test-slave-j2vl krb5kdc[4542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:05.732950 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.723732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:36107 (local address 127.8.141.254:37861)
0809 23:13:05.723797 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.723799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.723971 (+ 172us) server_negotiation.cc:407] Connection header received
0809 23:13:05.724830 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.724831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.724842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.724914 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:05.726003 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.726518 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.727557 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.727839 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.727924 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.730291 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:05.730293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:05.730293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:05.730302 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:05.731521 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.731933 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.731934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.731935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.732002 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.732261 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.732262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.732262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.732397 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:05.732477 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.732757 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.732845 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:13:05.733191 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.723621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:37861 (local address 127.8.141.195:36107)
0809 23:13:05.723820 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.723868 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.724650 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.725016 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.725023 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.725694 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:05.725906 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.725911 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.726622 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.726623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.727457 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.727459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.727940 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.727941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.728068 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.728643 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:05.728652 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:05.730188 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:05.731608 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:05.731617 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:05.731619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:05.731848 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:05.732071 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:05.732072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:05.732073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:05.732185 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:05.732466 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:05.732474 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:05.732620 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.732736 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.732920 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260809 23:13:05.733872 5380 heartbeater.cc:347] Connected to a master server at 127.8.141.254:37861
I20260809 23:13:05.733968 5380 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:05.734124 5380 heartbeater.cc:511] Master 127.8.141.254:37861 requested a full tablet report, sending...
I20260809 23:13:05.734498 4607 ts_manager.cc:194] Registered new tserver with Master: 09d8ffe9364240c8a57c9b6708eeef19 (127.8.141.195:42029)
I20260809 23:13:05.734861 4607 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:36107
I20260809 23:13:05.735744 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:13:05.743939 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.737089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56906 (local address 127.8.141.254:37861)
0809 23:13:05.737166 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.737169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.737288 (+ 119us) server_negotiation.cc:407] Connection header received
0809 23:13:05.737390 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.737391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.737401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.737450 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:05.738047 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.738528 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.739273 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.739477 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.739555 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.740789 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:05.740790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:05.740791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:05.740799 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:05.742440 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.742933 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.742935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.742936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.743006 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:05.743244 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:05.743245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:05.743246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:05.743432 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:05.743561 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.743704 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.743800 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:13:05.746150 4607 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:13:05.747789 4607 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:13:05.755221 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.750537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:44023 (local address 127.0.0.1:59058)
0809 23:13:05.750904 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.750914 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.750957 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.751334 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.751338 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.751341 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:05.751432 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.751434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.752605 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.752607 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.753818 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.753820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.754899 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.754900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.754961 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.754971 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.755064 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.755123 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":4,"thread_start_us":95,"threads_started":1}
I20260809 23:13:05.755234 5407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.750850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.8.141.193:44023)
0809 23:13:05.751080 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.751086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.751098 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:13:05.751147 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.751149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.751168 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.751299 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:05.751557 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.752490 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.753919 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.754731 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.754801 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.754868 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.755080 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.755127 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":98,"threads_started":1}
I20260809 23:13:05.755508 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.750537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41777 (local address 127.0.0.1:49550)
0809 23:13:05.750736 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.750782 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.750891 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.751735 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.751736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.751739 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:05.751793 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.751794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.753246 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.753248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.754271 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.754273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.755202 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.755203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.755274 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.755279 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.755332 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.755387 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
I20260809 23:13:05.756084 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.750704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.8.141.194:41777)
0809 23:13:05.751487 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.751491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.751502 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:13:05.751545 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.751547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.751563 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.751644 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:05.751902 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.753110 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.754372 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.755032 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.755109 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.755164 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.755686 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.755737 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":92,"threads_started":1}
I20260809 23:13:05.756333 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.750766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42029 (local address 127.0.0.1:55468)
0809 23:13:05.751215 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.751226 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.751331 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.751722 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.751723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.751725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:05.751775 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.751778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.753268 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.753269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.754646 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.754649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.756073 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.756074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.756146 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.756152 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.756214 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.756263 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":84,"threads_started":1}
I20260809 23:13:05.756309 4822 tablet_service.cc:1467] Processing CreateTablet for tablet f111eab8b6044c1a920fa2f18e7aec73 (DEFAULT_TABLE table=test-table [id=118679d3a34d4949978eccb7ecef7f4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:05.756628 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.751008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.8.141.195:42029)
0809 23:13:05.751238 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.751270 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.751473 (+ 203us) server_negotiation.cc:407] Connection header received
0809 23:13:05.751525 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.751528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.751548 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.751633 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:05.751869 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.753146 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.754774 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.755881 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.755969 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.756033 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.756521 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.756561 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":106,"threads_started":1}
I20260809 23:13:05.756729 4822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f111eab8b6044c1a920fa2f18e7aec73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:05.756793 5045 tablet_service.cc:1467] Processing CreateTablet for tablet f111eab8b6044c1a920fa2f18e7aec73 (DEFAULT_TABLE table=test-table [id=118679d3a34d4949978eccb7ecef7f4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:05.757130 5045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f111eab8b6044c1a920fa2f18e7aec73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:05.757252 5276 tablet_service.cc:1467] Processing CreateTablet for tablet f111eab8b6044c1a920fa2f18e7aec73 (DEFAULT_TABLE table=test-table [id=118679d3a34d4949978eccb7ecef7f4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:05.757644 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f111eab8b6044c1a920fa2f18e7aec73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:05.760594 5409 tablet_bootstrap.cc:492] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Bootstrap starting.
I20260809 23:13:05.760993 5410 tablet_bootstrap.cc:492] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f: Bootstrap starting.
I20260809 23:13:05.761428 5409 tablet_bootstrap.cc:654] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:05.761525 5412 tablet_bootstrap.cc:492] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Bootstrap starting.
I20260809 23:13:05.761777 5410 tablet_bootstrap.cc:654] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:05.762043 5409 log.cc:824] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:05.762300 5412 tablet_bootstrap.cc:654] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:05.762452 5410 log.cc:824] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:05.762883 5412 log.cc:824] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:05.762972 5409 tablet_bootstrap.cc:492] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: No bootstrap required, opened a new log
I20260809 23:13:05.763047 5409 ts_tablet_manager.cc:1397] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 23:13:05.763320 5410 tablet_bootstrap.cc:492] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f: No bootstrap required, opened a new log
I20260809 23:13:05.763396 5410 ts_tablet_manager.cc:1397] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:05.763727 5412 tablet_bootstrap.cc:492] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: No bootstrap required, opened a new log
I20260809 23:13:05.763805 5412 ts_tablet_manager.cc:1397] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:05.764673 5409 raft_consensus.cc:348] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.764850 5409 raft_consensus.cc:374] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:05.764899 5409 raft_consensus.cc:729] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d8ffe9364240c8a57c9b6708eeef19, State: Initialized, Role: FOLLOWER
I20260809 23:13:05.765025 5409 consensus_queue.cc:260] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.765359 5380 heartbeater.cc:503] Master 127.8.141.254:37861 was elected leader, sending a full tablet report...
I20260809 23:13:05.765383 5409 ts_tablet_manager.cc:1428] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 23:13:05.765671 5410 raft_consensus.cc:348] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.765875 5410 raft_consensus.cc:374] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:05.765921 5410 raft_consensus.cc:729] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a356b9120a994f5eb00b2f0141a91b7f, State: Initialized, Role: FOLLOWER
I20260809 23:13:05.766036 5410 consensus_queue.cc:260] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.766119 5412 raft_consensus.cc:348] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.766341 5410 ts_tablet_manager.cc:1428] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:13:05.766336 5412 raft_consensus.cc:374] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:05.766383 5412 raft_consensus.cc:729] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8d2817b48dc4791937334311b8c87d4, State: Initialized, Role: FOLLOWER
I20260809 23:13:05.766458 4926 heartbeater.cc:503] Master 127.8.141.254:37861 was elected leader, sending a full tablet report...
I20260809 23:13:05.766527 5412 consensus_queue.cc:260] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.766819 5412 ts_tablet_manager.cc:1428] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:13:05.766958 5152 heartbeater.cc:503] Master 127.8.141.254:37861 was elected leader, sending a full tablet report...
W20260809 23:13:05.787014 5154 tablet.cc:2369] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:13:05.860606 5419 raft_consensus.cc:482] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:13:05.860742 5419 raft_consensus.cc:504] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.861119 5419 leader_election.cc:288] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8d2817b48dc4791937334311b8c87d4 (127.8.141.194:41777), 09d8ffe9364240c8a57c9b6708eeef19 (127.8.141.195:42029)
I20260809 23:13:05.865389 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.861477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:57791 (local address 127.8.141.194:41777)
0809 23:13:05.861546 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.861549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.861562 (+ 13us) server_negotiation.cc:407] Connection header received
0809 23:13:05.861595 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.861596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.861608 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.861707 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:05.861923 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.862683 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.864302 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.864923 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.865010 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.865020 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.865243 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.865285 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:13:05.865384 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.861630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:52809 (local address 127.8.141.195:42029)
0809 23:13:05.861703 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.861706 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.861715 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:05.861772 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.861773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.861782 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.861835 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:05.862009 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.862710 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.864301 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.864930 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.865010 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.865020 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.865243 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.865285 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:13:05.865442 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.861335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41777 (local address 127.8.141.193:57791)
0809 23:13:05.861432 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.861445 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.861494 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.861730 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.861731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.861733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:05.861840 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.861842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.862793 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.862794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.864176 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.864180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.865041 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.865042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.865152 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.865154 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.865221 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.865276 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":89}
I20260809 23:13:05.865450 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.861515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:42029 (local address 127.8.141.193:52809)
0809 23:13:05.861657 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.861666 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:05.861715 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:05.861855 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:05.861856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:05.861858 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:05.861932 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.861933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.862793 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.862794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.864176 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:05.864180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.865041 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.865042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:05.865146 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.865151 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:13:05.865221 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:13:05.865279 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":72,"thread_start_us":87,"threads_started":1}
I20260809 23:13:05.865902 5077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f111eab8b6044c1a920fa2f18e7aec73" candidate_uuid: "a356b9120a994f5eb00b2f0141a91b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8d2817b48dc4791937334311b8c87d4" is_pre_election: true
I20260809 23:13:05.865909 5303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f111eab8b6044c1a920fa2f18e7aec73" candidate_uuid: "a356b9120a994f5eb00b2f0141a91b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d8ffe9364240c8a57c9b6708eeef19" is_pre_election: true
I20260809 23:13:05.866076 5303 raft_consensus.cc:2454] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a356b9120a994f5eb00b2f0141a91b7f in term 0.
I20260809 23:13:05.866076 5077 raft_consensus.cc:2454] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a356b9120a994f5eb00b2f0141a91b7f in term 0.
I20260809 23:13:05.866353 4751 leader_election.cc:302] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a356b9120a994f5eb00b2f0141a91b7f, d8d2817b48dc4791937334311b8c87d4; no voters:
I20260809 23:13:05.866492 5419 raft_consensus.cc:2781] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:13:05.866590 5419 raft_consensus.cc:482] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:13:05.866631 5419 raft_consensus.cc:3037] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:05.867431 5419 raft_consensus.cc:504] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.867588 5419 leader_election.cc:288] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [CANDIDATE]: Term 1 election: Requested vote from peers d8d2817b48dc4791937334311b8c87d4 (127.8.141.194:41777), 09d8ffe9364240c8a57c9b6708eeef19 (127.8.141.195:42029)
I20260809 23:13:05.867786 5077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f111eab8b6044c1a920fa2f18e7aec73" candidate_uuid: "a356b9120a994f5eb00b2f0141a91b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8d2817b48dc4791937334311b8c87d4"
I20260809 23:13:05.867790 5303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f111eab8b6044c1a920fa2f18e7aec73" candidate_uuid: "a356b9120a994f5eb00b2f0141a91b7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d8ffe9364240c8a57c9b6708eeef19"
I20260809 23:13:05.867887 5077 raft_consensus.cc:3037] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:05.867908 5303 raft_consensus.cc:3037] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:05.868868 5077 raft_consensus.cc:2454] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a356b9120a994f5eb00b2f0141a91b7f in term 1.
I20260809 23:13:05.868868 5303 raft_consensus.cc:2454] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a356b9120a994f5eb00b2f0141a91b7f in term 1.
I20260809 23:13:05.869093 4751 leader_election.cc:302] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a356b9120a994f5eb00b2f0141a91b7f, d8d2817b48dc4791937334311b8c87d4; no voters:
I20260809 23:13:05.869194 5419 raft_consensus.cc:2781] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:05.869412 5419 raft_consensus.cc:686] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [term 1 LEADER]: Becoming Leader. State: Replica: a356b9120a994f5eb00b2f0141a91b7f, State: Running, Role: LEADER
I20260809 23:13:05.869643 5419 consensus_queue.cc:237] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } }
I20260809 23:13:05.870713 4607 catalog_manager.cc:5495] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f reported cstate change: term changed from 0 to 1, leader changed from <none> to a356b9120a994f5eb00b2f0141a91b7f (127.8.141.193). New cstate: current_term: 1 leader_uuid: "a356b9120a994f5eb00b2f0141a91b7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356b9120a994f5eb00b2f0141a91b7f" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 } health_report { overall_health: UNKNOWN } } }
W20260809 23:13:05.875334 4927 tablet.cc:2369] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:13:05.901826 5407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.898937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.8.141.193:44023)
0809 23:13:05.899004 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:05.899007 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:05.899016 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:05.899067 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:05.899068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:05.899096 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:05.899188 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:05.899343 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.899888 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.900873 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:05.901093 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:05.901178 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:05.901405 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:05.901521 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:05.901624 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:05.901670 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 23:13:05.901694 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:13:05.904258 5077 raft_consensus.cc:1261] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 1 FOLLOWER]: Refusing update from remote peer a356b9120a994f5eb00b2f0141a91b7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:05.904258 5303 raft_consensus.cc:1261] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 1 FOLLOWER]: Refusing update from remote peer a356b9120a994f5eb00b2f0141a91b7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:05.904670 5424 consensus_queue.cc:1035] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d8ffe9364240c8a57c9b6708eeef19" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 42029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:05.904830 5419 consensus_queue.cc:1035] T f111eab8b6044c1a920fa2f18e7aec73 P a356b9120a994f5eb00b2f0141a91b7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8d2817b48dc4791937334311b8c87d4" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:05.909752 4607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:05.909878 4607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:05.910660 4607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7acfdb2dde0490a9e3d2113060ca5f0: Sending DeleteTablet for 3 replicas of tablet f111eab8b6044c1a920fa2f18e7aec73
I20260809 23:13:05.910974 4822 tablet_service.cc:1514] Processing DeleteTablet for tablet f111eab8b6044c1a920fa2f18e7aec73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:05 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:59058
I20260809 23:13:05.911027 5276 tablet_service.cc:1514] Processing DeleteTablet for tablet f111eab8b6044c1a920fa2f18e7aec73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:05 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:55468
I20260809 23:13:05.911222 5045 tablet_service.cc:1514] Processing DeleteTablet for tablet f111eab8b6044c1a920fa2f18e7aec73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:05 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:49550
I20260809 23:13:05.911330 5447 tablet_replica.cc:330] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: stopping tablet replica
I20260809 23:13:05.911435 5447 raft_consensus.cc:2229] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:05.911513 5448 tablet_replica.cc:330] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: stopping tablet replica
I20260809 23:13:05.911696 5448 raft_consensus.cc:2229] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:05.911757 5447 raft_consensus.cc:2258] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:05.911839 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 4716
I20260809 23:13:05.911868 5448 raft_consensus.cc:2258] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:05.912485 5448 ts_tablet_manager.cc:1905] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:05.914139 5448 ts_tablet_manager.cc:1918] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:05.914203 5448 log.cc:1196] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/f111eab8b6044c1a920fa2f18e7aec73
I20260809 23:13:05.914453 5448 ts_tablet_manager.cc:1939] T f111eab8b6044c1a920fa2f18e7aec73 P d8d2817b48dc4791937334311b8c87d4: Deleting consensus metadata
I20260809 23:13:05.915553 5447 ts_tablet_manager.cc:1905] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:05.919958 4585 catalog_manager.cc:4841] TS d8d2817b48dc4791937334311b8c87d4 (127.8.141.194:41777): tablet f111eab8b6044c1a920fa2f18e7aec73 (table test-table [id=118679d3a34d4949978eccb7ecef7f4d]) successfully deleted
I20260809 23:13:05.920265 5447 ts_tablet_manager.cc:1918] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:05.920375 5447 log.cc:1196] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/f111eab8b6044c1a920fa2f18e7aec73
I20260809 23:13:05.920668 5447 ts_tablet_manager.cc:1939] T f111eab8b6044c1a920fa2f18e7aec73 P 09d8ffe9364240c8a57c9b6708eeef19: Deleting consensus metadata
I20260809 23:13:05.921187 4585 catalog_manager.cc:4841] TS 09d8ffe9364240c8a57c9b6708eeef19 (127.8.141.195:42029): tablet f111eab8b6044c1a920fa2f18e7aec73 (table test-table [id=118679d3a34d4949978eccb7ecef7f4d]) successfully deleted
W20260809 23:13:05.924980 4587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:44023 (error 108)
I20260809 23:13:05.925187 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 4933
I20260809 23:13:05.933898 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:05.933706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:44023 (local address 127.0.0.1:59072)
0809 23:13:05.933790 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:05.933858 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:44023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 23:13:05.934118 4587 catalog_manager.cc:4584] TS a356b9120a994f5eb00b2f0141a91b7f (127.8.141.193:44023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f111eab8b6044c1a920fa2f18e7aec73: Network error: Client connection negotiation failed: client connection to 127.8.141.193:44023: connect: Connection refused (error 111)
I20260809 23:13:05.938139 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 5159
I20260809 23:13:05.950515 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 4560
2026-08-09T23:13:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2475 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T23:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:05Z Disabled control of system clock
I20260809 23:13:05.991808 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42787
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:36659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:06.082158 5460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:06.082391 5460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:06.082412 5460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:06.083598 5460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:06.083644 5460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:06.083657 5460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:06.083670 5460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:06.083727 5460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:06.085099 5460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:06.085402 5460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:06.085729 5460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:06.091126 5465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:06.091219 5472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:06.091261 5467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:06.091873 5460 server_base.cc:1034] running on GCE node
I20260809 23:13:06.092134 5460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:06.092408 5460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:06.093575 5460 hybrid_clock.cc:648] HybridClock initialized: now 1786317186093557 us; error 29 us; skew 500 ppm
I20260809 23:13:06.094990 5460 webserver.cc:460] Webserver started at http://127.8.141.254:45567/ using document root <none> and password file <none>
I20260809 23:13:06.095276 5460 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:06.095356 5460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:06.095520 5460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:06.096668 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "4379971af2d44351a95ebfbe86c41b2e"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "bc7029ff31586d5c69a0437de0e03b12"
server_key_iv: "74201b5aa53fb480502542cff09db5b7"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.097044 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "4379971af2d44351a95ebfbe86c41b2e"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "bc7029ff31586d5c69a0437de0e03b12"
server_key_iv: "74201b5aa53fb480502542cff09db5b7"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.098588 5460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:06.099535 5483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.099746 5460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:06.099858 5460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "4379971af2d44351a95ebfbe86c41b2e"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "bc7029ff31586d5c69a0437de0e03b12"
server_key_iv: "74201b5aa53fb480502542cff09db5b7"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.099953 5460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:06.113637 5460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:06.113982 5460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:06.114142 5460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:06.118136 5460 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42787
I20260809 23:13:06.118197 5562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42787 every 8 connection(s)
I20260809 23:13:06.118547 5460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:06.119680 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:06.122368 5565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e: Bootstrap starting.
I20260809 23:13:06.123044 5565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:06.123394 5565 log.cc:824] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:06.124159 5565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e: No bootstrap required, opened a new log
I20260809 23:13:06.126077 5565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4379971af2d44351a95ebfbe86c41b2e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42787 } }
I20260809 23:13:06.126231 5565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:06.126256 5565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4379971af2d44351a95ebfbe86c41b2e, State: Initialized, Role: FOLLOWER
I20260809 23:13:06.126443 5565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4379971af2d44351a95ebfbe86c41b2e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42787 } }
I20260809 23:13:06.126518 5565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:06.126542 5565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:06.126600 5565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:06.127250 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 5460
I20260809 23:13:06.127382 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:06.127419 5565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4379971af2d44351a95ebfbe86c41b2e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42787 } }
I20260809 23:13:06.127547 5565 leader_election.cc:302] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4379971af2d44351a95ebfbe86c41b2e; no voters:
I20260809 23:13:06.127583 8759 external_mini_cluster.cc:1428] Setting key 965a03d51b724776438a6957caca1138
I20260809 23:13:06.127765 5565 leader_election.cc:288] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:06.127846 5573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:06.128072 5573 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [term 1 LEADER]: Becoming Leader. State: Replica: 4379971af2d44351a95ebfbe86c41b2e, State: Running, Role: LEADER
I20260809 23:13:06.128257 5573 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4379971af2d44351a95ebfbe86c41b2e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42787 } }
I20260809 23:13:06.128124 5565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:06.128659 5575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4379971af2d44351a95ebfbe86c41b2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4379971af2d44351a95ebfbe86c41b2e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42787 } } }
I20260809 23:13:06.128743 5577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4379971af2d44351a95ebfbe86c41b2e. Latest consensus state: current_term: 1 leader_uuid: "4379971af2d44351a95ebfbe86c41b2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4379971af2d44351a95ebfbe86c41b2e" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42787 } } }
I20260809 23:13:06.128804 5577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:06.128755 5575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:06.129433 5581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:06.130007 5581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:06.131659 5581 catalog_manager.cc:1347] Generated new cluster ID: 645748cb9aab4655bfacf33eb4f33dcb
I20260809 23:13:06.131719 5581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:06.134608 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.128304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.8.141.254:42787)
0809 23:13:06.128957 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.128967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.129002 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:13:06.129121 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.129128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.129203 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.129407 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:06.129638 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.131163 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.132268 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.132925 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.133060 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.133820 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:06.133824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:06.133824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:06.134055 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:06.134100 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:06.134115 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:06.134184 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.134285 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.134367 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
I20260809 23:13:06.166822 5581 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:06.167398 5581 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:06.170125 5581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e: Generated new TSK 0
I20260809 23:13:06.170351 5581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:06.175568 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42787
--builtin_ntp_servers=127.8.141.212:36659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:06.264482 5607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:06.264724 5607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:06.264765 5607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:06.266117 5607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:06.266165 5607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:06.266193 5607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:13:06.267764 5607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:06.268065 5607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:06.268409 5607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:06.273921 5620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:06.274008 5618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:06.273921 5616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:06.274137 5607 server_base.cc:1034] running on GCE node
I20260809 23:13:06.274379 5607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:06.274648 5607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:06.275794 5607 hybrid_clock.cc:648] HybridClock initialized: now 1786317186275772 us; error 37 us; skew 500 ppm
I20260809 23:13:06.277149 5607 webserver.cc:460] Webserver started at http://127.8.141.193:35893/ using document root <none> and password file <none>
I20260809 23:13:06.277456 5607 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:06.277509 5607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:06.277661 5607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:06.278796 5607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "f3bb6fdd271343eebf12871d7763af45"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "3657e2aa8ce4c7ef11708dc5bcfe0721"
server_key_iv: "328472aff42ee090a7a8c88bf998c3d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.279176 5607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "f3bb6fdd271343eebf12871d7763af45"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "3657e2aa8ce4c7ef11708dc5bcfe0721"
server_key_iv: "328472aff42ee090a7a8c88bf998c3d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.280591 5607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:06.281551 5629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.281804 5607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:06.281912 5607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "f3bb6fdd271343eebf12871d7763af45"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "3657e2aa8ce4c7ef11708dc5bcfe0721"
server_key_iv: "328472aff42ee090a7a8c88bf998c3d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.282018 5607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:06.292399 5607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:06.292708 5607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:06.292855 5607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:06.293094 5607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:06.293486 5607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:06.293541 5607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.293586 5607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:06.293612 5607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.299551 5607 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:36753
I20260809 23:13:06.299654 5797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:36753 every 8 connection(s)
I20260809 23:13:06.299983 5607 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:13:06.302172 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 5607
I20260809 23:13:06.302290 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:13:06.302489 8759 external_mini_cluster.cc:1428] Setting key 1c7dc880a6ceedc53b5aa7ef96d42d0b
I20260809 23:13:06.303666 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42787
--builtin_ntp_servers=127.8.141.212:36659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:13:06.306243 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.300884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:42503 (local address 127.8.141.254:42787)
0809 23:13:06.300954 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.300956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.301197 (+ 241us) server_negotiation.cc:407] Connection header received
0809 23:13:06.301814 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.301816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.301829 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.301886 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:06.302737 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.303458 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.304582 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.304898 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.304984 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.305889 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:06.305891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:06.305891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:06.305897 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:06.305916 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:06.305922 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:06.305987 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.306110 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.306145 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 23:13:06.306449 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.300861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42787 (local address 127.8.141.193:42503)
0809 23:13:06.301083 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.301117 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.301692 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.301985 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.301989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.301997 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:06.302625 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.302631 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.303566 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.303567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.304495 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.304498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.304997 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.304998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.305107 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.305738 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:06.305739 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:06.305776 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:06.305799 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:06.306014 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:06.306022 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:06.306024 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.306097 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.306184 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260809 23:13:06.306969 5800 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42787
I20260809 23:13:06.307066 5800 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:06.307272 5800 heartbeater.cc:511] Master 127.8.141.254:42787 requested a full tablet report, sending...
I20260809 23:13:06.307816 5509 ts_manager.cc:194] Registered new tserver with Master: f3bb6fdd271343eebf12871d7763af45 (127.8.141.193:36753)
I20260809 23:13:06.308462 5509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.193:42503
W20260809 23:13:06.388352 5805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:06.388604 5805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:06.388643 5805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:06.389909 5805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:06.389974 5805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:06.390017 5805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:13:06.391535 5805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:06.391805 5805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:06.392112 5805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:13:06.397540 5805 server_base.cc:1034] running on GCE node
W20260809 23:13:06.397680 5812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:06.397671 5813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:06.397680 5816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:06.398061 5805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:06.398350 5805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:06.399559 5805 hybrid_clock.cc:648] HybridClock initialized: now 1786317186399547 us; error 41 us; skew 500 ppm
I20260809 23:13:06.401046 5805 webserver.cc:460] Webserver started at http://127.8.141.194:34769/ using document root <none> and password file <none>
I20260809 23:13:06.401413 5805 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:06.401500 5805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:06.401671 5805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:06.402945 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "ef0b8a7fe740495586224a031d02fb00"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "f8cedf331bd156bf01e4a132f6334313"
server_key_iv: "37222cce59aa476e7ea68236987eac52"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.403393 5805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "ef0b8a7fe740495586224a031d02fb00"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "f8cedf331bd156bf01e4a132f6334313"
server_key_iv: "37222cce59aa476e7ea68236987eac52"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.404899 5805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:06.405790 5824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.405972 5805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.406076 5805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "ef0b8a7fe740495586224a031d02fb00"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "f8cedf331bd156bf01e4a132f6334313"
server_key_iv: "37222cce59aa476e7ea68236987eac52"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.406167 5805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:06.426038 5805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:06.426402 5805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:06.426568 5805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:06.426837 5805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:06.427201 5805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:06.427258 5805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.427307 5805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:06.427341 5805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.434083 5805 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:45969
I20260809 23:13:06.434160 5999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:45969 every 8 connection(s)
I20260809 23:13:06.434510 5805 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:13:06.439850 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.435354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:58019 (local address 127.8.141.254:42787)
0809 23:13:06.435422 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.435424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.435657 (+ 233us) server_negotiation.cc:407] Connection header received
0809 23:13:06.436149 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.436150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.436160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.436223 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:06.436966 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.437536 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.438564 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.438744 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.438794 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.439609 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:06.439610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:06.439611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:06.439615 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:06.439628 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:06.439632 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:06.439674 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.439758 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.439783 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:13:06.440070 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.435305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42787 (local address 127.8.141.194:58019)
0809 23:13:06.435522 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.435566 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.436064 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.436274 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.436277 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.436286 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:06.436825 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.436830 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.437625 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.437626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.438486 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.438489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.438838 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.438838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.438949 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.439479 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:06.439479 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:06.439508 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:06.439530 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:06.439687 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:06.439692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:06.439694 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.439748 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.439847 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260809 23:13:06.440366 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 5805
I20260809 23:13:06.440500 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:13:06.440701 8759 external_mini_cluster.cc:1428] Setting key d2e4f51931fb7c952bce8b18dc196939
I20260809 23:13:06.440675 6000 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42787
I20260809 23:13:06.440786 6000 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:06.441007 6000 heartbeater.cc:511] Master 127.8.141.254:42787 requested a full tablet report, sending...
I20260809 23:13:06.441460 5509 ts_manager.cc:194] Registered new tserver with Master: ef0b8a7fe740495586224a031d02fb00 (127.8.141.194:45969)
I20260809 23:13:06.441716 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42787
--builtin_ntp_servers=127.8.141.212:36659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:13:06.441926 5509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.194:58019
W20260809 23:13:06.529237 6005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:06.529567 6005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:06.529609 6005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:06.530953 6005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:06.531018 6005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:06.531064 6005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:13:06.532512 6005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:42787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:06.532821 6005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:06.533154 6005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:06.538658 6011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:06.538650 6015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:06.538760 6005 server_base.cc:1034] running on GCE node
W20260809 23:13:06.538865 6010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:06.539072 6005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:06.539301 6005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:06.540464 6005 hybrid_clock.cc:648] HybridClock initialized: now 1786317186540438 us; error 45 us; skew 500 ppm
I20260809 23:13:06.541766 6005 webserver.cc:460] Webserver started at http://127.8.141.195:36619/ using document root <none> and password file <none>
I20260809 23:13:06.541996 6005 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:06.542039 6005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:06.542138 6005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:06.543198 6005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "3b41e6d655234ea3a7b10f20fd2b2db9"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "b28c912de59c9e9ec594980b96d07f67"
server_key_iv: "5db1988e7062c254362bfa03c5eb8260"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.543511 6005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "3b41e6d655234ea3a7b10f20fd2b2db9"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "b28c912de59c9e9ec594980b96d07f67"
server_key_iv: "5db1988e7062c254362bfa03c5eb8260"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.544855 6005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:13:06.545699 6024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.545928 6005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:06.546039 6005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "3b41e6d655234ea3a7b10f20fd2b2db9"
format_stamp: "Formatted at 2026-08-09 23:13:06 on dist-test-slave-j2vl"
server_key: "b28c912de59c9e9ec594980b96d07f67"
server_key_iv: "5db1988e7062c254362bfa03c5eb8260"
server_key_version: "encryptionkey@0"
I20260809 23:13:06.546120 6005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:06.556649 6005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:06.557022 6005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:06.557142 6005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:06.557493 6005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:06.557858 6005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:06.557900 6005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.557924 6005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:06.557964 6005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:06.563755 6005 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:46317
I20260809 23:13:06.563867 6207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:46317 every 8 connection(s)
I20260809 23:13:06.564193 6005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:13:06.567548 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 6005
I20260809 23:13:06.567660 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:13:06.567889 8759 external_mini_cluster.cc:1428] Setting key 98a6bb07cfb6b4b4efbeb221bcfa554d
I20260809 23:13:06.570520 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.565055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:51771 (local address 127.8.141.254:42787)
0809 23:13:06.565139 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.565142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.565396 (+ 254us) server_negotiation.cc:407] Connection header received
0809 23:13:06.565995 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.565997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.566012 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.566062 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:06.566804 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.567461 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.568906 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.569209 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.569296 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.570195 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:06.570197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:06.570197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:06.570202 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:06.570218 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:06.570222 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:06.570295 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.570407 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.570440 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 23:13:06.570747 6212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.565010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:42787 (local address 127.8.141.195:51771)
0809 23:13:06.565220 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.565283 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.565877 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.566141 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.566144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.566153 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:06.566693 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.566698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.567565 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.567566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.568802 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.568806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.569318 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.569319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.569515 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.570046 (+ 531us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:06.570046 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:06.570074 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:06.570096 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:06.570299 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:06.570306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:06.570309 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.570409 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.570512 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
I20260809 23:13:06.571175 6208 heartbeater.cc:347] Connected to a master server at 127.8.141.254:42787
I20260809 23:13:06.571269 6208 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:06.571485 6208 heartbeater.cc:511] Master 127.8.141.254:42787 requested a full tablet report, sending...
I20260809 23:13:06.571838 5509 ts_manager.cc:194] Registered new tserver with Master: 3b41e6d655234ea3a7b10f20fd2b2db9 (127.8.141.195:46317)
I20260809 23:13:06.572228 5509 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.195:51771
I20260809 23:13:06.579798 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:13:06.584599 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.581491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52036 (local address 127.8.141.254:42787)
0809 23:13:06.581571 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.581578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.581616 (+ 38us) server_negotiation.cc:407] Connection header received
0809 23:13:06.581685 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.581685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.581698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.581754 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:06.581952 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.582454 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.583268 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.583478 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.583544 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.584292 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:06.584293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:06.584293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:06.584298 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:06.584312 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:06.584316 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:06.584366 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.584436 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.584466 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 23:13:06.586325 5509 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:13:06.587981 5509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:13:06.594935 6225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.590268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:45969 (local address 127.0.0.1:50808)
0809 23:13:06.590688 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.590697 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.590733 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.591013 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.591017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.591019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:06.591094 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.591096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.592414 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.592415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.593555 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.593558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.594568 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.594569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.594635 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.594646 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.594765 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.594833 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":10,"thread_start_us":70,"threads_started":1}
I20260809 23:13:06.594944 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.590560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50808 (local address 127.8.141.194:45969)
0809 23:13:06.590792 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.590796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.590805 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:06.590875 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.590877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.590896 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.590992 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:06.591193 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.592322 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.593805 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.594479 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.594548 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.594644 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.594810 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.594853 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260809 23:13:06.595551 6223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.590265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:46317 (local address 127.0.0.1:54136)
0809 23:13:06.590468 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.590510 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.590578 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.591345 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.591345 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.591347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:06.591383 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.591384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.592612 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.592613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.593574 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.593577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.595322 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.595323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.595389 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.595393 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.595448 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.595486 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":112,"threads_started":1}
I20260809 23:13:06.595817 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.590348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54136 (local address 127.8.141.195:46317)
0809 23:13:06.591135 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.591138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.591148 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:13:06.591186 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.591187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.591202 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.591276 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:06.591471 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.592528 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.593708 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.594467 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.594548 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.594644 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.595664 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.595714 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":81,"threads_started":1}
I20260809 23:13:06.595754 5896 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1e192eb66744d09265ecb33dd7dae8 (DEFAULT_TABLE table=test-table [id=b1b97273803a4773bac0d43bdb046305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:06.596215 5896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1e192eb66744d09265ecb33dd7dae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:06.596326 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.590298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:36753 (local address 127.0.0.1:34190)
0809 23:13:06.590810 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.590817 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.590853 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.591256 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.591257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.591259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:06.591305 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.591307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.592190 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.592191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.593555 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.593558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.595850 (+ 2292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.595851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.596131 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.596136 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.596213 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.596257 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":5,"thread_start_us":97,"threads_started":1}
I20260809 23:13:06.596354 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.590537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34190 (local address 127.8.141.193:36753)
0809 23:13:06.590741 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.590746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.590883 (+ 137us) server_negotiation.cc:407] Connection header received
0809 23:13:06.590928 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.590930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.590945 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.591135 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:06.591392 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.592099 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.594684 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.595653 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.595718 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.595816 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.596232 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.596274 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":89,"threads_started":1}
I20260809 23:13:06.596506 6106 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1e192eb66744d09265ecb33dd7dae8 (DEFAULT_TABLE table=test-table [id=b1b97273803a4773bac0d43bdb046305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:06.596841 6106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1e192eb66744d09265ecb33dd7dae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:06.597005 5700 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1e192eb66744d09265ecb33dd7dae8 (DEFAULT_TABLE table=test-table [id=b1b97273803a4773bac0d43bdb046305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:06.597245 5700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1e192eb66744d09265ecb33dd7dae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:06.599979 6231 tablet_bootstrap.cc:492] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Bootstrap starting.
I20260809 23:13:06.600309 6235 tablet_bootstrap.cc:492] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Bootstrap starting.
I20260809 23:13:06.600818 6231 tablet_bootstrap.cc:654] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:06.600888 6235 tablet_bootstrap.cc:654] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:06.601130 6231 log.cc:824] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:06.601156 6235 log.cc:824] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:06.601504 6234 tablet_bootstrap.cc:492] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45: Bootstrap starting.
I20260809 23:13:06.602176 6231 tablet_bootstrap.cc:492] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: No bootstrap required, opened a new log
I20260809 23:13:06.602253 6234 tablet_bootstrap.cc:654] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:06.602294 6231 ts_tablet_manager.cc:1397] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:13:06.602577 6234 log.cc:824] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:06.603415 6234 tablet_bootstrap.cc:492] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45: No bootstrap required, opened a new log
I20260809 23:13:06.603487 6234 ts_tablet_manager.cc:1397] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:13:06.605111 6231 raft_consensus.cc:348] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.605315 6231 raft_consensus.cc:374] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:06.605384 6231 raft_consensus.cc:729] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0b8a7fe740495586224a031d02fb00, State: Initialized, Role: FOLLOWER
I20260809 23:13:06.605516 6231 consensus_queue.cc:260] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.605791 6231 ts_tablet_manager.cc:1428] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 23:13:06.605850 6000 heartbeater.cc:503] Master 127.8.141.254:42787 was elected leader, sending a full tablet report...
I20260809 23:13:06.606251 6235 tablet_bootstrap.cc:492] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: No bootstrap required, opened a new log
I20260809 23:13:06.606184 6234 raft_consensus.cc:348] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.606328 6235 ts_tablet_manager.cc:1397] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260809 23:13:06.606346 6234 raft_consensus.cc:374] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:06.606385 6234 raft_consensus.cc:729] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bb6fdd271343eebf12871d7763af45, State: Initialized, Role: FOLLOWER
I20260809 23:13:06.606508 6234 consensus_queue.cc:260] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.606791 6234 ts_tablet_manager.cc:1428] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 23:13:06.606870 5800 heartbeater.cc:503] Master 127.8.141.254:42787 was elected leader, sending a full tablet report...
I20260809 23:13:06.608666 6235 raft_consensus.cc:348] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.608803 6235 raft_consensus.cc:374] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:06.608836 6235 raft_consensus.cc:729] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b41e6d655234ea3a7b10f20fd2b2db9, State: Initialized, Role: FOLLOWER
I20260809 23:13:06.608979 6235 consensus_queue.cc:260] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.609299 6235 ts_tablet_manager.cc:1428] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:13:06.609506 6208 heartbeater.cc:503] Master 127.8.141.254:42787 was elected leader, sending a full tablet report...
W20260809 23:13:06.685134 6001 tablet.cc:2369] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:13:06.693105 6243 raft_consensus.cc:482] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:13:06.693231 6243 raft_consensus.cc:504] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.693658 6243 leader_election.cc:288] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef0b8a7fe740495586224a031d02fb00 (127.8.141.194:45969), 3b41e6d655234ea3a7b10f20fd2b2db9 (127.8.141.195:46317)
I20260809 23:13:06.697893 6230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.694008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:50447 (local address 127.8.141.194:45969)
0809 23:13:06.694057 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.694060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.694290 (+ 230us) server_negotiation.cc:407] Connection header received
0809 23:13:06.694347 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.694348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.694366 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.694439 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:06.694559 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.695282 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.696867 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.697465 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.697543 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.697553 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.697773 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.697816 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 23:13:06.697917 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.694065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:36889 (local address 127.8.141.195:46317)
0809 23:13:06.694122 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.694124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.694136 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:13:06.694200 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.694200 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.694209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.694287 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:06.694473 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.695282 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.696867 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.697453 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.697543 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.697553 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.697773 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.697816 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 23:13:06.697933 5804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.693954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:46317 (local address 127.8.141.193:36889)
0809 23:13:06.694041 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.694054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.694104 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.694301 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.694302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.694303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:06.694386 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.694388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.695383 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.695385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.696761 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.696765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.697583 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.697584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.697679 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.697686 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.697757 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.697811 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":61}
I20260809 23:13:06.697943 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.693954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:45969 (local address 127.8.141.193:50447)
0809 23:13:06.694210 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.694219 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:06.694292 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:06.694424 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:06.694425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:06.694426 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:06.694488 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.694489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.695383 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.695385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.696776 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:06.696778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.697583 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.697584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:06.697679 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.697686 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 23:13:06.697757 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 23:13:06.697810 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":127,"thread_start_us":117,"threads_started":1}
I20260809 23:13:06.698354 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e192eb66744d09265ecb33dd7dae8" candidate_uuid: "f3bb6fdd271343eebf12871d7763af45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" is_pre_election: true
I20260809 23:13:06.698567 6139 raft_consensus.cc:2454] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bb6fdd271343eebf12871d7763af45 in term 0.
I20260809 23:13:06.698529 5924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e192eb66744d09265ecb33dd7dae8" candidate_uuid: "f3bb6fdd271343eebf12871d7763af45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0b8a7fe740495586224a031d02fb00" is_pre_election: true
I20260809 23:13:06.698663 5924 raft_consensus.cc:2454] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bb6fdd271343eebf12871d7763af45 in term 0.
I20260809 23:13:06.698912 5633 leader_election.cc:302] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b41e6d655234ea3a7b10f20fd2b2db9, f3bb6fdd271343eebf12871d7763af45; no voters:
I20260809 23:13:06.699077 6243 raft_consensus.cc:2781] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:13:06.699143 6243 raft_consensus.cc:482] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:13:06.699168 6243 raft_consensus.cc:3037] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:06.699952 6243 raft_consensus.cc:504] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.700124 6243 leader_election.cc:288] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [CANDIDATE]: Term 1 election: Requested vote from peers ef0b8a7fe740495586224a031d02fb00 (127.8.141.194:45969), 3b41e6d655234ea3a7b10f20fd2b2db9 (127.8.141.195:46317)
I20260809 23:13:06.700290 5924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e192eb66744d09265ecb33dd7dae8" candidate_uuid: "f3bb6fdd271343eebf12871d7763af45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0b8a7fe740495586224a031d02fb00"
I20260809 23:13:06.700340 6139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1e192eb66744d09265ecb33dd7dae8" candidate_uuid: "f3bb6fdd271343eebf12871d7763af45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9"
I20260809 23:13:06.700372 5924 raft_consensus.cc:3037] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:06.700443 6139 raft_consensus.cc:3037] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:06.701162 5924 raft_consensus.cc:2454] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bb6fdd271343eebf12871d7763af45 in term 1.
I20260809 23:13:06.701289 6139 raft_consensus.cc:2454] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bb6fdd271343eebf12871d7763af45 in term 1.
I20260809 23:13:06.701377 5632 leader_election.cc:302] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef0b8a7fe740495586224a031d02fb00, f3bb6fdd271343eebf12871d7763af45; no voters:
I20260809 23:13:06.701483 6243 raft_consensus.cc:2781] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:06.701648 6243 raft_consensus.cc:686] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [term 1 LEADER]: Becoming Leader. State: Replica: f3bb6fdd271343eebf12871d7763af45, State: Running, Role: LEADER
I20260809 23:13:06.701767 6243 consensus_queue.cc:237] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } }
I20260809 23:13:06.702800 5508 catalog_manager.cc:5495] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3bb6fdd271343eebf12871d7763af45 (127.8.141.193). New cstate: current_term: 1 leader_uuid: "f3bb6fdd271343eebf12871d7763af45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3bb6fdd271343eebf12871d7763af45" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 36753 } health_report { overall_health: HEALTHY } } }
I20260809 23:13:06.711892 6229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.709436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.8.141.193:36753)
0809 23:13:06.709487 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:06.709489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:06.709496 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:13:06.709528 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:06.709529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:06.709548 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:06.709643 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:06.709826 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.710293 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.711064 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:06.711258 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:06.711334 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:06.711518 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:06.711622 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:06.711710 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:06.711751 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 23:13:06.711772 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:13:06.713997 5924 raft_consensus.cc:1261] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 1 FOLLOWER]: Refusing update from remote peer f3bb6fdd271343eebf12871d7763af45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:06.714016 6139 raft_consensus.cc:1261] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 1 FOLLOWER]: Refusing update from remote peer f3bb6fdd271343eebf12871d7763af45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:06.714393 6249 consensus_queue.cc:1035] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b41e6d655234ea3a7b10f20fd2b2db9" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 46317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:06.714545 6243 consensus_queue.cc:1035] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef0b8a7fe740495586224a031d02fb00" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 45969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:06.718076 6256 mvcc.cc:205] Tried to move back new op lower bound from 7316755196777156608 to 7316755196731154432. Current Snapshot: MvccSnapshot[applied={T|T < 7316755196777156608}]
I20260809 23:13:06.719846 5508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:06.720083 5508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:06.720896 5508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4379971af2d44351a95ebfbe86c41b2e: Sending DeleteTablet for 3 replicas of tablet 6a1e192eb66744d09265ecb33dd7dae8
I20260809 23:13:06.721267 5896 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1e192eb66744d09265ecb33dd7dae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:06 UTC) from {username='slave'} at 127.0.0.1:50808
I20260809 23:13:06.721285 5700 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1e192eb66744d09265ecb33dd7dae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:06 UTC) from {username='slave'} at 127.0.0.1:34190
I20260809 23:13:06.721302 6106 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1e192eb66744d09265ecb33dd7dae8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:06 UTC) from {username='slave'} at 127.0.0.1:54136
I20260809 23:13:06.721617 6269 tablet_replica.cc:330] T 6a1e192eb66744d09265ecb33dd7dae8 P f3bb6fdd271343eebf12871d7763af45: stopping tablet replica
I20260809 23:13:06.721653 6266 tablet_replica.cc:330] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: stopping tablet replica
I20260809 23:13:06.721768 6268 tablet_replica.cc:330] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: stopping tablet replica
I20260809 23:13:06.721776 6266 raft_consensus.cc:2229] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:06.721886 6266 raft_consensus.cc:2258] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:06.721984 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 5607
I20260809 23:13:06.722117 6268 raft_consensus.cc:2229] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:06.725284 6268 raft_consensus.cc:2258] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:06.725420 6266 ts_tablet_manager.cc:1905] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:06.727466 6266 ts_tablet_manager.cc:1918] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:06.727551 6266 log.cc:1196] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/6a1e192eb66744d09265ecb33dd7dae8
I20260809 23:13:06.727813 6266 ts_tablet_manager.cc:1939] T 6a1e192eb66744d09265ecb33dd7dae8 P ef0b8a7fe740495586224a031d02fb00: Deleting consensus metadata
I20260809 23:13:06.731936 6268 ts_tablet_manager.cc:1905] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:06.734647 6268 ts_tablet_manager.cc:1918] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:06.734856 6268 log.cc:1196] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/6a1e192eb66744d09265ecb33dd7dae8
I20260809 23:13:06.735280 6268 ts_tablet_manager.cc:1939] T 6a1e192eb66744d09265ecb33dd7dae8 P 3b41e6d655234ea3a7b10f20fd2b2db9: Deleting consensus metadata
I20260809 23:13:06.736002 5485 catalog_manager.cc:4841] TS 3b41e6d655234ea3a7b10f20fd2b2db9 (127.8.141.195:46317): tablet 6a1e192eb66744d09265ecb33dd7dae8 (table test-table [id=b1b97273803a4773bac0d43bdb046305]) successfully deleted
I20260809 23:13:06.736984 5484 catalog_manager.cc:4841] TS ef0b8a7fe740495586224a031d02fb00 (127.8.141.194:45969): tablet 6a1e192eb66744d09265ecb33dd7dae8 (table test-table [id=b1b97273803a4773bac0d43bdb046305]) successfully deleted
W20260809 23:13:06.744287 5487 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:36753 (error 108)
I20260809 23:13:06.744522 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 5805
I20260809 23:13:06.749055 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:06.747379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:36753 (local address 127.0.0.1:34204)
0809 23:13:06.748934 (+ 1555us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:06.749004 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:36753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1522}
W20260809 23:13:06.749236 5487 catalog_manager.cc:4584] TS f3bb6fdd271343eebf12871d7763af45 (127.8.141.193:36753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a1e192eb66744d09265ecb33dd7dae8: Network error: Client connection negotiation failed: client connection to 127.8.141.193:36753: connect: Connection refused (error 111)
I20260809 23:13:06.760483 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 6005
I20260809 23:13:06.775744 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 5460
2026-08-09T23:13:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (819 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:06 dist-test-slave-j2vl krb5kdc[6278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:06 dist-test-slave-j2vl krb5kdc[6278](info): set up 2 sockets
Aug 09 23:13:06 dist-test-slave-j2vl krb5kdc[6278](info): commencing operation
krb5kdc: starting...
W20260809 23:13:08.079095 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:08.217655 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36827
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:42685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:36827
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:08.306325 6303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:08.306553 6303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:08.306577 6303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:08.307837 6303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:08.307885 6303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:08.307900 6303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:08.307914 6303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:08.307925 6303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:08.309602 6303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:36827
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:36827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:08.309899 6303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:08.310160 6303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:08.315716 6323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:08.315703 6315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:08.315910 6303 server_base.cc:1034] running on GCE node
W20260809 23:13:08.315703 6317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:08.316314 6303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:08.316648 6303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:08.317818 6303 hybrid_clock.cc:648] HybridClock initialized: now 1786317188317810 us; error 42 us; skew 500 ppm
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:08.320513 6303 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:08.321002 6303 webserver.cc:460] Webserver started at http://127.8.141.254:34131/ using document root <none> and password file <none>
I20260809 23:13:08.321218 6303 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:08.321259 6303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:08.321416 6303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:08.322465 6303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "28d7d280ca5d4f9f9664ce8a6c201967"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "7de1e3ebd963e7450c8ea71a26925356"
server_key_iv: "dec3d91544657be7000ba4d8617d7671"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.322758 6303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "28d7d280ca5d4f9f9664ce8a6c201967"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "7de1e3ebd963e7450c8ea71a26925356"
server_key_iv: "dec3d91544657be7000ba4d8617d7671"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.324083 6303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:13:08.324800 6336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.324994 6303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:08.325090 6303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "28d7d280ca5d4f9f9664ce8a6c201967"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "7de1e3ebd963e7450c8ea71a26925356"
server_key_iv: "dec3d91544657be7000ba4d8617d7671"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.325148 6303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:08.334830 6303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:08.336673 6303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:08.336788 6303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:08.340775 6303 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:36827
I20260809 23:13:08.340826 6428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:36827 every 8 connection(s)
I20260809 23:13:08.341199 6303 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:08.342130 6431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:08.343815 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 6303
I20260809 23:13:08.343914 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:08.344121 8759 external_mini_cluster.cc:1428] Setting key 57cbc9c1f349cd6f26a48d300cb8797c
I20260809 23:13:08.344707 6431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967: Bootstrap starting.
I20260809 23:13:08.345484 6431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:08.345824 6431 log.cc:824] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:08.346614 6431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967: No bootstrap required, opened a new log
I20260809 23:13:08.348445 6431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36827 } }
I20260809 23:13:08.348584 6431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:08.348605 6431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28d7d280ca5d4f9f9664ce8a6c201967, State: Initialized, Role: FOLLOWER
I20260809 23:13:08.348753 6431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36827 } }
I20260809 23:13:08.348829 6431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:08.348872 6431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:08.348927 6431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:08.349661 6431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36827 } }
I20260809 23:13:08.349789 6431 leader_election.cc:302] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28d7d280ca5d4f9f9664ce8a6c201967; no voters:
I20260809 23:13:08.350013 6431 leader_election.cc:288] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:08.350059 6440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:08.350224 6440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [term 1 LEADER]: Becoming Leader. State: Replica: 28d7d280ca5d4f9f9664ce8a6c201967, State: Running, Role: LEADER
I20260809 23:13:08.350323 6431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:08.350366 6440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36827 } }
I20260809 23:13:08.350951 6440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28d7d280ca5d4f9f9664ce8a6c201967. Latest consensus state: current_term: 1 leader_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36827 } } }
I20260809 23:13:08.351219 6440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:08.351145 6442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28d7d280ca5d4f9f9664ce8a6c201967" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 36827 } } }
I20260809 23:13:08.351672 6442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967 [sys.catalog]: This master's current role is: LEADER
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:08.351802 6452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:08.352696 6452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:08.354584 6452 catalog_manager.cc:1347] Generated new cluster ID: 210c4ade7e564caaa046bbbef29c0dd7
I20260809 23:13:08.354647 6452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:08.357277 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.344725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35730 (local address 127.8.141.254:36827)
0809 23:13:08.344944 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.344954 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.344984 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:13:08.345671 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.345682 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.345749 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.345918 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:08.346800 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.347691 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.348493 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.349137 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.349262 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.352461 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:08.352467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:08.352467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:08.352483 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:08.354908 (+ 2425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.355408 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.355411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.355412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.355559 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.355780 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.355781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.355781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.356176 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:08.356389 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.356744 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.356995 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":96,"threads_started":1}
I20260809 23:13:08.366118 6452 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:08.366847 6452 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:08.371120 6452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967: Generated new TSK 0
I20260809 23:13:08.371346 6452 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:13:08.408985 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36827
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:42685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:08.533911 6470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:08.534190 6470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:08.534238 6470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:08.535633 6470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:08.535712 6470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:08.535763 6470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:13:08.537645 6470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:08.537984 6470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:08.538249 6470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:08.544137 6486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:08.544137 6480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:08.544276 6478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:08.544291 6470 server_base.cc:1034] running on GCE node
I20260809 23:13:08.544698 6470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:08.544986 6470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:08.546183 6470 hybrid_clock.cc:648] HybridClock initialized: now 1786317188546147 us; error 31 us; skew 500 ppm
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:08.554697 6470 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:13:08.555364 6470 webserver.cc:460] Webserver started at http://127.8.141.193:40301/ using document root <none> and password file <none>
I20260809 23:13:08.555620 6470 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:08.555696 6470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:08.555853 6470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:08.557204 6470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "e4c41bafbb924794abc300f35dbea18a"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "021741f2a57aa7f08bc7fa812b3b5b5f"
server_key_iv: "fac2287ca78f33deec78bb7043e60c1d"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.557678 6470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "e4c41bafbb924794abc300f35dbea18a"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "021741f2a57aa7f08bc7fa812b3b5b5f"
server_key_iv: "fac2287ca78f33deec78bb7043e60c1d"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.559345 6470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:08.560403 6497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.560771 6470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:08.561020 6470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "e4c41bafbb924794abc300f35dbea18a"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "021741f2a57aa7f08bc7fa812b3b5b5f"
server_key_iv: "fac2287ca78f33deec78bb7043e60c1d"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.561122 6470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:08.585844 6470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:08.589021 6470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:08.589215 6470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:08.589535 6470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:08.589928 6470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:08.589967 6470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.590016 6470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:08.590045 6470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.596925 6470 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:41261
I20260809 23:13:08.597015 6684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:41261 every 8 connection(s)
I20260809 23:13:08.597440 6470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:08.607419 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 6470
I20260809 23:13:08.607561 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:13:08.607817 8759 external_mini_cluster.cc:1428] Setting key 283d6bd88f508ddaa1edd0ab01117175
I20260809 23:13:08.608439 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.598363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:40641 (local address 127.8.141.254:36827)
0809 23:13:08.598496 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.598499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.598670 (+ 171us) server_negotiation.cc:407] Connection header received
0809 23:13:08.599296 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.599298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.599312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.599397 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:08.600470 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.600995 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.602148 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.602470 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.602606 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.605115 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:08.605116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:08.605117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:08.605125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:08.606648 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.607093 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.607094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.607095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.607154 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.607505 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.607506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.607506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.607728 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:08.607805 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.608187 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.608328 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 23:13:08.608575 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.598279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36827 (local address 127.8.141.193:40641)
0809 23:13:08.598522 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:08.598574 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:08.599137 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:08.599481 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:08.599485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:08.600138 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:08.600359 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.600364 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.601106 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.601107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.602046 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.602049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.602640 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.602641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.602781 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.603421 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:08.603436 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:08.605002 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:08.606741 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:08.606751 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:08.606753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:08.607018 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:08.607233 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:08.607234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:08.607234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:08.607400 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:08.607809 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:08.607818 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:08.608012 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 23:13:08.608126 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 23:13:08.608334 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":99,"threads_started":1}
I20260809 23:13:08.609267 6685 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36827
I20260809 23:13:08.609411 6685 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:08.609640 6685 heartbeater.cc:511] Master 127.8.141.254:36827 requested a full tablet report, sending...
I20260809 23:13:08.610303 6363 ts_manager.cc:194] Registered new tserver with Master: e4c41bafbb924794abc300f35dbea18a (127.8.141.193:41261)
I20260809 23:13:08.611085 6363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:40641
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:13:08.641708 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36827
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:42685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:08.726011 6695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:08.726224 6695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:08.726249 6695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:08.727409 6695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:08.727460 6695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:08.727487 6695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:13:08.729029 6695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:08.729327 6695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:08.729604 6695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:13:08.735047 6695 server_base.cc:1034] running on GCE node
W20260809 23:13:08.735047 6705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:08.735073 6702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:08.735203 6703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:08.735502 6695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:08.735791 6695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:08.736966 6695 hybrid_clock.cc:648] HybridClock initialized: now 1786317188736959 us; error 38 us; skew 500 ppm
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:08.739800 6695 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:13:08.740293 6695 webserver.cc:460] Webserver started at http://127.8.141.194:46419/ using document root <none> and password file <none>
I20260809 23:13:08.740556 6695 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:08.740626 6695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:08.740777 6695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:08.741911 6695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "c01253637cdd45819cc1e1eff315b26b"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "4e0206f1e544fcd8519e64286c751727"
server_key_iv: "3bb7b94c070c4ff21b33e23229bd7239"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.742273 6695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "c01253637cdd45819cc1e1eff315b26b"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "4e0206f1e544fcd8519e64286c751727"
server_key_iv: "3bb7b94c070c4ff21b33e23229bd7239"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.743628 6695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:08.744407 6717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.744612 6695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:08.744720 6695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "c01253637cdd45819cc1e1eff315b26b"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "4e0206f1e544fcd8519e64286c751727"
server_key_iv: "3bb7b94c070c4ff21b33e23229bd7239"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.744796 6695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:08.755627 6695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:08.758044 6695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:08.758216 6695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:08.758456 6695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:08.758781 6695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:08.758838 6695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.758875 6695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:08.758904 6695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.764868 6695 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:46231
I20260809 23:13:08.764926 6894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:46231 every 8 connection(s)
I20260809 23:13:08.765250 6695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:13:08.767724 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 6695
I20260809 23:13:08.767886 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:13:08.768080 8759 external_mini_cluster.cc:1428] Setting key 64282cdbcf6ed6f27bb44e02465f3d0d
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:08.776093 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.766185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:40497 (local address 127.8.141.254:36827)
0809 23:13:08.766258 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.766261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.766548 (+ 287us) server_negotiation.cc:407] Connection header received
0809 23:13:08.767048 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.767049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.767061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.767120 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:08.768127 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.768800 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.770036 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.770266 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.770358 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.773082 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:08.773084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:08.773084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:08.773095 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:08.774620 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.775071 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.775072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.775073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.775136 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.775405 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.775406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.775406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.775530 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:08.775603 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.775896 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.775994 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 23:13:08.776328 6899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.766177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36827 (local address 127.8.141.194:40497)
0809 23:13:08.766421 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:08.766459 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:08.766940 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:08.767169 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:08.767173 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:08.767754 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:08.768023 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.768027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.768899 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.768900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.769817 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.769820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.770372 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.770373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.770515 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.771093 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:08.771103 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:08.772969 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:08.774738 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:08.774748 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:08.774750 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:08.774991 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:08.775207 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:08.775208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:08.775208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:08.775329 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:08.775606 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:08.775614 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:08.775760 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 23:13:08.775872 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 23:13:08.776045 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":122,"threads_started":1}
I20260809 23:13:08.776831 6895 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36827
I20260809 23:13:08.776953 6895 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:08.777172 6895 heartbeater.cc:511] Master 127.8.141.254:36827 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
I20260809 23:13:08.777598 6363 ts_manager.cc:194] Registered new tserver with Master: c01253637cdd45819cc1e1eff315b26b (127.8.141.194:46231)
I20260809 23:13:08.777995 6363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:40497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:13:08.798044 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36827
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:42685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:08.886252 6904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:08.886477 6904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:08.886504 6904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:08.887645 6904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:08.887682 6904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:08.887708 6904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:13:08.889156 6904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:36827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:08.889431 6904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:08.889653 6904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:08.895753 6913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:08.895870 6904 server_base.cc:1034] running on GCE node
W20260809 23:13:08.895761 6914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:08.895772 6917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:08.896258 6904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:08.896550 6904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:08.897693 6904 hybrid_clock.cc:648] HybridClock initialized: now 1786317188897672 us; error 45 us; skew 500 ppm
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:08.900532 6904 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:13:08.901006 6904 webserver.cc:460] Webserver started at http://127.8.141.195:41369/ using document root <none> and password file <none>
I20260809 23:13:08.901268 6904 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:08.901350 6904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:08.901539 6904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:08.902634 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "2c3183cb21e147a49d68ab863557a73f"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "0d7164d03b69e1285ea4689d4fde7dce"
server_key_iv: "d99ca4b9810c073c1c114325aafd8acf"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.902998 6904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "2c3183cb21e147a49d68ab863557a73f"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "0d7164d03b69e1285ea4689d4fde7dce"
server_key_iv: "d99ca4b9810c073c1c114325aafd8acf"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.904457 6904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:08.905269 6928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.905485 6904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:08.905596 6904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "2c3183cb21e147a49d68ab863557a73f"
format_stamp: "Formatted at 2026-08-09 23:13:08 on dist-test-slave-j2vl"
server_key: "0d7164d03b69e1285ea4689d4fde7dce"
server_key_iv: "d99ca4b9810c073c1c114325aafd8acf"
server_key_version: "encryptionkey@0"
I20260809 23:13:08.905673 6904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:08.918707 6904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:08.921087 6904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:08.921269 6904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:08.921540 6904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:08.921893 6904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:08.921954 6904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.921998 6904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:08.922045 6904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:08.928025 6904 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:41711
I20260809 23:13:08.928069 7115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:41711 every 8 connection(s)
I20260809 23:13:08.928459 6904 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:13:08.934423 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 6904
I20260809 23:13:08.934509 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:13:08.934689 8759 external_mini_cluster.cc:1428] Setting key 275b4efa1143cb02748e42b765f457e4
Aug 09 23:13:08 dist-test-slave-j2vl krb5kdc[6278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317188, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:08.938228 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.929187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:54137 (local address 127.8.141.254:36827)
0809 23:13:08.929252 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.929255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.929537 (+ 282us) server_negotiation.cc:407] Connection header received
0809 23:13:08.930074 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.930076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.930087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.930147 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:08.931066 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.931595 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.932629 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.932827 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.932915 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.935529 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:08.935531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:08.935531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:08.935539 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:08.936695 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.937132 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.937133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.937133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.937197 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.937526 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.937527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.937528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.937647 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:08.937719 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.938025 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.938136 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:13:08.938428 7120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.929187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:36827 (local address 127.8.141.195:54137)
0809 23:13:08.929395 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:08.929446 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:08.929919 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:08.930197 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:08.930200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:08.930769 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:08.930966 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.930971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.931695 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.931696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.932527 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.932529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.932936 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.932937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.933046 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.933686 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:08.933694 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:08.935416 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:08.936817 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:08.936824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:08.936826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:08.937050 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:08.937274 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:08.937275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:08.937276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:08.937447 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:08.937724 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:08.937732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:08.937877 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 23:13:08.937989 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 23:13:08.938153 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":73,"threads_started":1}
I20260809 23:13:08.939111 7116 heartbeater.cc:347] Connected to a master server at 127.8.141.254:36827
I20260809 23:13:08.939222 7116 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:08.939414 7116 heartbeater.cc:511] Master 127.8.141.254:36827 requested a full tablet report, sending...
I20260809 23:13:08.939862 6363 ts_manager.cc:194] Registered new tserver with Master: 2c3183cb21e147a49d68ab863557a73f (127.8.141.195:41711)
I20260809 23:13:08.940244 6363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:54137
I20260809 23:13:08.946873 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:13:08.955793 6436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.948691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.8.141.254:36827)
0809 23:13:08.948833 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.948835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.948842 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:13:08.948864 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.948865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.948874 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.948936 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:08.949641 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.950171 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.951062 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.951282 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.951366 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.952899 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:08.952901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:08.952901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:08.952910 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:08.954429 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.954906 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.954907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.954908 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.954970 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:08.955206 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:08.955207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:08.955208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:08.955363 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:08.955487 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.955601 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.955666 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260809 23:13:08.957680 6363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:13:08.959383 6363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:13:08.966651 7136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.962033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41711 (local address 127.0.0.1:33794)
0809 23:13:08.962403 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:08.962448 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:08.962517 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:08.962902 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:08.962906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:08.962909 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:08.963028 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.963031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.964141 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.964142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.965393 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.965396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.966318 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.966319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.966414 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.966429 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 23:13:08.966504 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:13:08.966561 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":90,"threads_started":1}
I20260809 23:13:08.966683 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.962223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33794 (local address 127.8.141.195:41711)
0809 23:13:08.962666 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.962670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.962679 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:08.962722 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.962724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.962737 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.962821 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:08.963151 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.964027 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.965493 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.966218 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.966304 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.966365 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.966539 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.966595 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":107,"threads_started":1}
I20260809 23:13:08.966823 7147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.962415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49832 (local address 127.8.141.193:41261)
0809 23:13:08.962611 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.962618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.963049 (+ 431us) server_negotiation.cc:407] Connection header received
0809 23:13:08.963103 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.963106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.963124 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.963205 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:08.963447 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.964615 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.965804 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.966330 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.966401 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.966461 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.966695 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.966745 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":113,"threads_started":1}
I20260809 23:13:08.967067 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.962260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41261 (local address 127.0.0.1:49832)
0809 23:13:08.962954 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:08.962966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:08.963027 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:08.963308 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:08.963309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:08.963311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:08.963360 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.963361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.964730 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.964732 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.965711 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.965713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.966496 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.966497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.966550 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.966554 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:08.966600 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 23:13:08.966634 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":76,"threads_started":1}
I20260809 23:13:08.967267 7138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.962033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:46231 (local address 127.0.0.1:53400)
0809 23:13:08.962422 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:08.962448 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:08.962517 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:08.963313 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:08.963315 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:08.963317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:08.963366 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.963368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.964664 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.964665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.965736 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:08.965738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.966707 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.966708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:08.967001 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.967006 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:13:08.967067 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:13:08.967186 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":4,"thread_start_us":68,"threads_started":1}
I20260809 23:13:08.967581 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:08.962187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53400 (local address 127.8.141.194:46231)
0809 23:13:08.963162 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:08.963167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:08.963177 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:13:08.963216 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:08.963218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:08.963233 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:08.963304 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:08.963452 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.964556 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.965838 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:08.966541 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:08.966606 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:08.966668 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:08.967105 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 23:13:08.967159 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":932,"thread_start_us":105,"threads_started":1}
I20260809 23:13:08.967548 7003 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 (DEFAULT_TABLE table=test-table [id=17505af8c28d4855b9012f71c1b45f50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:08.967963 7003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:08.968263 6787 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 (DEFAULT_TABLE table=test-table [id=17505af8c28d4855b9012f71c1b45f50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:08.968580 6787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:08.971634 7148 tablet_bootstrap.cc:492] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Bootstrap starting.
I20260809 23:13:08.972059 6584 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 (DEFAULT_TABLE table=test-table [id=17505af8c28d4855b9012f71c1b45f50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:08.972440 6584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:08.972446 7148 tablet_bootstrap.cc:654] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:08.972944 7148 log.cc:824] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:08.973600 7149 tablet_bootstrap.cc:492] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Bootstrap starting.
I20260809 23:13:08.973834 7148 tablet_bootstrap.cc:492] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: No bootstrap required, opened a new log
I20260809 23:13:08.973901 7148 ts_tablet_manager.cc:1397] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:08.974303 7149 tablet_bootstrap.cc:654] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:08.974900 7149 log.cc:824] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:08.975379 7153 tablet_bootstrap.cc:492] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a: Bootstrap starting.
I20260809 23:13:08.975628 7149 tablet_bootstrap.cc:492] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: No bootstrap required, opened a new log
I20260809 23:13:08.975674 7149 ts_tablet_manager.cc:1397] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:08.976022 7153 tablet_bootstrap.cc:654] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:08.976217 7148 raft_consensus.cc:348] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:08.976441 7148 raft_consensus.cc:374] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:08.976506 7148 raft_consensus.cc:729] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3183cb21e147a49d68ab863557a73f, State: Initialized, Role: FOLLOWER
I20260809 23:13:08.976591 7153 log.cc:824] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:08.976629 7148 consensus_queue.cc:260] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:08.976939 7148 ts_tablet_manager.cc:1428] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:13:08.976966 7116 heartbeater.cc:503] Master 127.8.141.254:36827 was elected leader, sending a full tablet report...
I20260809 23:13:08.977437 7149 raft_consensus.cc:348] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:08.977541 7153 tablet_bootstrap.cc:492] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a: No bootstrap required, opened a new log
I20260809 23:13:08.977603 7149 raft_consensus.cc:374] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:08.977641 7149 raft_consensus.cc:729] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01253637cdd45819cc1e1eff315b26b, State: Initialized, Role: FOLLOWER
I20260809 23:13:08.977666 7153 ts_tablet_manager.cc:1397] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:08.977754 7149 consensus_queue.cc:260] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:08.978046 7149 ts_tablet_manager.cc:1428] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:13:08.978183 6895 heartbeater.cc:503] Master 127.8.141.254:36827 was elected leader, sending a full tablet report...
I20260809 23:13:08.979413 7153 raft_consensus.cc:348] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:08.979559 7153 raft_consensus.cc:374] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:08.979606 7153 raft_consensus.cc:729] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4c41bafbb924794abc300f35dbea18a, State: Initialized, Role: FOLLOWER
I20260809 23:13:08.979715 7153 consensus_queue.cc:260] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:08.979996 7153 ts_tablet_manager.cc:1428] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 23:13:08.980063 6685 heartbeater.cc:503] Master 127.8.141.254:36827 was elected leader, sending a full tablet report...
W20260809 23:13:09.015918 6897 tablet.cc:2369] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:13:09.056241 7156 raft_consensus.cc:482] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:13:09.056456 7156 raft_consensus.cc:504] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:09.056867 7156 leader_election.cc:288] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c01253637cdd45819cc1e1eff315b26b (127.8.141.194:46231), e4c41bafbb924794abc300f35dbea18a (127.8.141.193:41261)
I20260809 23:13:09.060206 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.057178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41261 (local address 127.8.141.195:33225)
0809 23:13:09.057370 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.057381 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.057443 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.057599 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.057600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.057601 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:09.057700 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.057702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.058415 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.058416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.059226 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.059228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.059922 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.059923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.060001 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.060005 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.060064 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.060095 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":126,"threads_started":1}
I20260809 23:13:09.060226 7147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.057268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:33225 (local address 127.8.141.193:41261)
0809 23:13:09.057365 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.057368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.057428 (+ 60us) server_negotiation.cc:407] Connection header received
0809 23:13:09.057517 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.057518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.057531 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.057585 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:09.057797 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.058308 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.059338 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.059812 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.059880 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.059889 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.060081 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.060132 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:13:09.060321 7120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.057063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:46231 (local address 127.8.141.195:35643)
0809 23:13:09.057152 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.057168 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.057257 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.057508 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.057508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.057510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:09.057594 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.057597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.058302 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.058304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.059327 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.059330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.060014 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.060015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.060121 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.060125 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.060193 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.060239 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 23:13:09.060447 7139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.057133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:35643 (local address 127.8.141.194:46231)
0809 23:13:09.057186 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.057189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.057219 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:13:09.057412 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.057413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.057427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.057489 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:09.057675 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.058204 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.059458 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.059920 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.059999 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.060009 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.060287 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.060354 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:13:09.060725 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0d9ad540bf4f4cb3a974b3c8dc1295" candidate_uuid: "2c3183cb21e147a49d68ab863557a73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01253637cdd45819cc1e1eff315b26b" is_pre_election: true
I20260809 23:13:09.060695 6611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0d9ad540bf4f4cb3a974b3c8dc1295" candidate_uuid: "2c3183cb21e147a49d68ab863557a73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4c41bafbb924794abc300f35dbea18a" is_pre_election: true
I20260809 23:13:09.060887 6611 raft_consensus.cc:2454] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3183cb21e147a49d68ab863557a73f in term 0.
I20260809 23:13:09.060887 6817 raft_consensus.cc:2454] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3183cb21e147a49d68ab863557a73f in term 0.
I20260809 23:13:09.061208 6932 leader_election.cc:302] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3183cb21e147a49d68ab863557a73f, e4c41bafbb924794abc300f35dbea18a; no voters:
I20260809 23:13:09.061388 7156 raft_consensus.cc:2781] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:13:09.061452 7156 raft_consensus.cc:482] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:13:09.061480 7156 raft_consensus.cc:3037] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:09.062206 7156 raft_consensus.cc:504] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:09.062359 7156 leader_election.cc:288] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [CANDIDATE]: Term 1 election: Requested vote from peers c01253637cdd45819cc1e1eff315b26b (127.8.141.194:46231), e4c41bafbb924794abc300f35dbea18a (127.8.141.193:41261)
I20260809 23:13:09.062570 6611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0d9ad540bf4f4cb3a974b3c8dc1295" candidate_uuid: "2c3183cb21e147a49d68ab863557a73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4c41bafbb924794abc300f35dbea18a"
I20260809 23:13:09.062628 6817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0d9ad540bf4f4cb3a974b3c8dc1295" candidate_uuid: "2c3183cb21e147a49d68ab863557a73f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c01253637cdd45819cc1e1eff315b26b"
I20260809 23:13:09.062652 6611 raft_consensus.cc:3037] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:09.062737 6817 raft_consensus.cc:3037] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:09.063455 6611 raft_consensus.cc:2454] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3183cb21e147a49d68ab863557a73f in term 1.
I20260809 23:13:09.063638 6932 leader_election.cc:302] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3183cb21e147a49d68ab863557a73f, e4c41bafbb924794abc300f35dbea18a; no voters:
I20260809 23:13:09.063766 6817 raft_consensus.cc:2454] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3183cb21e147a49d68ab863557a73f in term 1.
I20260809 23:13:09.063967 7156 raft_consensus.cc:2781] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:09.064193 7156 raft_consensus.cc:686] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 1 LEADER]: Becoming Leader. State: Replica: 2c3183cb21e147a49d68ab863557a73f, State: Running, Role: LEADER
I20260809 23:13:09.064311 7156 consensus_queue.cc:237] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } }
I20260809 23:13:09.065297 6362 catalog_manager.cc:5495] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c3183cb21e147a49d68ab863557a73f (127.8.141.195). New cstate: current_term: 1 leader_uuid: "2c3183cb21e147a49d68ab863557a73f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c3183cb21e147a49d68ab863557a73f" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 } health_report { overall_health: UNKNOWN } } }
I20260809 23:13:09.083989 7145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.081419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.8.141.195:41711)
0809 23:13:09.081468 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.081470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.081481 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:13:09.081537 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.081538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.081558 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.081613 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:09.081806 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.082289 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.083081 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.083280 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.083342 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.083536 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:09.083624 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:09.083700 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.083803 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.083813 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:13:09.086374 6817 raft_consensus.cc:1261] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 1 FOLLOWER]: Refusing update from remote peer 2c3183cb21e147a49d68ab863557a73f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:09.086372 6611 raft_consensus.cc:1261] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a [term 1 FOLLOWER]: Refusing update from remote peer 2c3183cb21e147a49d68ab863557a73f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:09.086798 7161 consensus_queue.cc:1035] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c01253637cdd45819cc1e1eff315b26b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 46231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:09.086968 7156 consensus_queue.cc:1035] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4c41bafbb924794abc300f35dbea18a" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:09.089581 7165 mvcc.cc:205] Tried to move back new op lower bound from 7316755206494154752 to 7316755206408220672. Current Snapshot: MvccSnapshot[applied={T|T < 7316755206494154752}]
I20260809 23:13:09.090260 7166 mvcc.cc:205] Tried to move back new op lower bound from 7316755206494154752 to 7316755206408220672. Current Snapshot: MvccSnapshot[applied={T|T < 7316755206494154752}]
I20260809 23:13:09.090437 7167 mvcc.cc:205] Tried to move back new op lower bound from 7316755206494154752 to 7316755206408220672. Current Snapshot: MvccSnapshot[applied={T|T < 7316755206494154752}]
I20260809 23:13:09.092175 6362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:09.092271 6362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:09.093060 6362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28d7d280ca5d4f9f9664ce8a6c201967: Sending DeleteTablet for 3 replicas of tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295
I20260809 23:13:09.093526 6787 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:09 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:53400
I20260809 23:13:09.093521 7003 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:09 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:33794
I20260809 23:13:09.093524 6584 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:09 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:49832
I20260809 23:13:09.093870 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 6470
I20260809 23:13:09.093871 7181 tablet_replica.cc:330] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P e4c41bafbb924794abc300f35dbea18a: stopping tablet replica
I20260809 23:13:09.104739 7180 tablet_replica.cc:330] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: stopping tablet replica
I20260809 23:13:09.104884 7180 raft_consensus.cc:2229] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:13:09.105001 7180 raft_consensus.cc:2258] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:09.105187 7179 tablet_replica.cc:330] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: stopping tablet replica
I20260809 23:13:09.105316 7179 raft_consensus.cc:2229] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:09.105468 7179 raft_consensus.cc:2258] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:09.105705 7180 ts_tablet_manager.cc:1905] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:09.105927 7179 ts_tablet_manager.cc:1905] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:09.107864 7179 ts_tablet_manager.cc:1918] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:09.107957 7179 log.cc:1196] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/8b0d9ad540bf4f4cb3a974b3c8dc1295
I20260809 23:13:09.108120 7180 ts_tablet_manager.cc:1918] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:09.108256 7179 ts_tablet_manager.cc:1939] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P c01253637cdd45819cc1e1eff315b26b: Deleting consensus metadata
I20260809 23:13:09.108263 7180 log.cc:1196] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/8b0d9ad540bf4f4cb3a974b3c8dc1295
I20260809 23:13:09.108803 7180 ts_tablet_manager.cc:1939] T 8b0d9ad540bf4f4cb3a974b3c8dc1295 P 2c3183cb21e147a49d68ab863557a73f: Deleting consensus metadata
I20260809 23:13:09.108944 6342 catalog_manager.cc:4841] TS c01253637cdd45819cc1e1eff315b26b (127.8.141.194:46231): tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 (table test-table [id=17505af8c28d4855b9012f71c1b45f50]) successfully deleted
I20260809 23:13:09.109433 6340 catalog_manager.cc:4841] TS 2c3183cb21e147a49d68ab863557a73f (127.8.141.195:41711): tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295 (table test-table [id=17505af8c28d4855b9012f71c1b45f50]) successfully deleted
W20260809 23:13:09.110625 6342 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:41261 (error 108)
I20260809 23:13:09.111207 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.111066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41261 (local address 127.0.0.1:49840)
0809 23:13:09.111126 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.111171 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:41261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 23:13:09.111325 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 6695
W20260809 23:13:09.120690 6342 catalog_manager.cc:4584] TS e4c41bafbb924794abc300f35dbea18a (127.8.141.193:41261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b0d9ad540bf4f4cb3a974b3c8dc1295: Network error: Client connection negotiation failed: client connection to 127.8.141.193:41261: connect: Connection refused (error 111)
I20260809 23:13:09.126660 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 6904
I20260809 23:13:09.140175 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 6303
2026-08-09T23:13:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2363 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T23:13:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:09Z Disabled control of system clock
I20260809 23:13:09.175695 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:46001
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:46001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:09.261981 7188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:09.262216 7188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:09.262239 7188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:09.263438 7188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:09.263486 7188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:09.263499 7188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:09.263512 7188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:09.263525 7188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:09.265033 7188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:46001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:46001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:09.265312 7188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:09.265682 7188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:09.270972 7196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:09.270995 7195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:09.271148 7199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:09.271281 7188 server_base.cc:1034] running on GCE node
I20260809 23:13:09.271597 7188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:09.271934 7188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:09.273125 7188 hybrid_clock.cc:648] HybridClock initialized: now 1786317189273105 us; error 43 us; skew 500 ppm
I20260809 23:13:09.274514 7188 webserver.cc:460] Webserver started at http://127.8.141.254:40029/ using document root <none> and password file <none>
I20260809 23:13:09.274807 7188 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:09.274884 7188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:09.275048 7188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:09.276160 7188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "68b84e266c3f4d3aabdbcdab4e16912d"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "3c4f1822a5f48a1a54db6c80a8794f73"
server_key_iv: "ea3db318e3e8b3b71e2f627a2d65d31d"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.276525 7188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "68b84e266c3f4d3aabdbcdab4e16912d"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "3c4f1822a5f48a1a54db6c80a8794f73"
server_key_iv: "ea3db318e3e8b3b71e2f627a2d65d31d"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.278066 7188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:09.278877 7208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.279066 7188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:09.279166 7188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "68b84e266c3f4d3aabdbcdab4e16912d"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "3c4f1822a5f48a1a54db6c80a8794f73"
server_key_iv: "ea3db318e3e8b3b71e2f627a2d65d31d"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.279254 7188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:09.295229 7188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:09.295621 7188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:09.295805 7188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:09.300252 7188 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:46001
I20260809 23:13:09.300292 7305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:46001 every 8 connection(s)
I20260809 23:13:09.300710 7188 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:09.300995 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 7188
I20260809 23:13:09.301117 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:09.301327 8759 external_mini_cluster.cc:1428] Setting key 166532088fdea0307ef146aa82536559
I20260809 23:13:09.302017 7311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:09.305200 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d: Bootstrap starting.
I20260809 23:13:09.305907 7311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:09.306247 7311 log.cc:824] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:09.307009 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d: No bootstrap required, opened a new log
I20260809 23:13:09.307384 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.301962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.8.141.254:46001)
0809 23:13:09.302216 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.302224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.302250 (+ 26us) server_negotiation.cc:407] Connection header received
0809 23:13:09.302698 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.302705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.302765 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.302881 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:09.303138 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.304012 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.305135 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.305735 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.305844 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.306684 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:09.306689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:09.306689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:09.306891 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:09.306927 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:09.306940 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:09.307024 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.307105 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.307196 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":106,"threads_started":1}
I20260809 23:13:09.309202 7311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46001 } }
I20260809 23:13:09.309392 7311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:09.309446 7311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b84e266c3f4d3aabdbcdab4e16912d, State: Initialized, Role: FOLLOWER
I20260809 23:13:09.309572 7311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46001 } }
I20260809 23:13:09.309653 7311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:09.309693 7311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:09.309741 7311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:09.310314 7311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46001 } }
I20260809 23:13:09.310436 7311 leader_election.cc:302] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68b84e266c3f4d3aabdbcdab4e16912d; no voters:
I20260809 23:13:09.310652 7311 leader_election.cc:288] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:09.310750 7320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:09.310950 7320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [term 1 LEADER]: Becoming Leader. State: Replica: 68b84e266c3f4d3aabdbcdab4e16912d, State: Running, Role: LEADER
I20260809 23:13:09.310990 7311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:09.311064 7320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46001 } }
I20260809 23:13:09.311450 7325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68b84e266c3f4d3aabdbcdab4e16912d. Latest consensus state: current_term: 1 leader_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46001 } } }
I20260809 23:13:09.311571 7325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:09.311445 7322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68b84e266c3f4d3aabdbcdab4e16912d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46001 } } }
I20260809 23:13:09.311749 7322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:09.311898 7332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:09.312463 7332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:09.314110 7332 catalog_manager.cc:1347] Generated new cluster ID: 0dc3a4efda214637808fa0cf63a12f39
I20260809 23:13:09.314167 7332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:09.345685 7332 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:09.346486 7332 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:09.351879 7332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d: Generated new TSK 0
I20260809 23:13:09.352128 7332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:09.360808 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:46001
--builtin_ntp_servers=127.8.141.212:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:09.448168 7353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:09.448395 7353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:09.448419 7353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:09.449692 7353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:09.449736 7353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:09.449770 7353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:13:09.451181 7353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:09.451447 7353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:09.451808 7353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:09.457194 7359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:09.457244 7353 server_base.cc:1034] running on GCE node
W20260809 23:13:09.457211 7360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:09.457212 7362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:09.457643 7353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:09.457896 7353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:09.459066 7353 hybrid_clock.cc:648] HybridClock initialized: now 1786317189459053 us; error 49 us; skew 500 ppm
I20260809 23:13:09.460559 7353 webserver.cc:460] Webserver started at http://127.8.141.193:33827/ using document root <none> and password file <none>
I20260809 23:13:09.460825 7353 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:09.460880 7353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:09.461016 7353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:09.462237 7353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "f347b74ada75447e8a8798eb042450bf"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "8a319b5ba87e80a63b90241406c501ee"
server_key_iv: "9cec68cbbe1e5a624fb408ecabc82eeb"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.462607 7353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "f347b74ada75447e8a8798eb042450bf"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "8a319b5ba87e80a63b90241406c501ee"
server_key_iv: "9cec68cbbe1e5a624fb408ecabc82eeb"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.464005 7353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:13:09.464816 7370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.465035 7353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:09.465157 7353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "f347b74ada75447e8a8798eb042450bf"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "8a319b5ba87e80a63b90241406c501ee"
server_key_iv: "9cec68cbbe1e5a624fb408ecabc82eeb"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.465241 7353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:09.490720 7353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:09.491091 7353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:09.491254 7353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:09.491515 7353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:09.491901 7353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:09.491959 7353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.492002 7353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:09.492039 7353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.498221 7353 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:43363
I20260809 23:13:09.498270 7565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:43363 every 8 connection(s)
I20260809 23:13:09.498615 7353 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:13:09.504204 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.499454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:60267 (local address 127.8.141.254:46001)
0809 23:13:09.499561 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.499564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.499810 (+ 246us) server_negotiation.cc:407] Connection header received
0809 23:13:09.500302 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.500307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.500323 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.500392 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:09.501129 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.501672 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.502686 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.502904 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.502964 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.503887 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:09.503888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:09.503889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:09.503894 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:09.503908 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:09.503913 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:09.503973 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.504101 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.504134 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 23:13:09.504460 7570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.499438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:46001 (local address 127.8.141.193:60267)
0809 23:13:09.499661 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.499704 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.500206 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.500461 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.500465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.500474 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:09.501037 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.501041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.501780 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.501781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.502598 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.502601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.502989 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.502990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.503102 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.503728 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:09.503728 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:09.503758 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:09.503782 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:09.504005 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:09.504013 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:09.504015 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.504090 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.504180 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":107,"threads_started":1}
I20260809 23:13:09.505160 7566 heartbeater.cc:347] Connected to a master server at 127.8.141.254:46001
I20260809 23:13:09.505307 7566 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:09.505561 7566 heartbeater.cc:511] Master 127.8.141.254:46001 requested a full tablet report, sending...
I20260809 23:13:09.506098 7236 ts_manager.cc:194] Registered new tserver with Master: f347b74ada75447e8a8798eb042450bf (127.8.141.193:43363)
I20260809 23:13:09.506675 7236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.193:60267
I20260809 23:13:09.507380 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 7353
I20260809 23:13:09.507481 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:13:09.507702 8759 external_mini_cluster.cc:1428] Setting key a01bb1718254aa8c11ba0e3e2cef2bc4
I20260809 23:13:09.508607 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:46001
--builtin_ntp_servers=127.8.141.212:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:09.594199 7571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:09.594432 7571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:09.594456 7571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:09.595633 7571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:09.595677 7571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:09.595705 7571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:13:09.597041 7571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:09.597256 7571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:09.597630 7571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:09.603044 7577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:09.603051 7582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:09.603120 7571 server_base.cc:1034] running on GCE node
W20260809 23:13:09.603051 7576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:09.603503 7571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:09.603776 7571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:09.604947 7571 hybrid_clock.cc:648] HybridClock initialized: now 1786317189604927 us; error 42 us; skew 500 ppm
I20260809 23:13:09.606282 7571 webserver.cc:460] Webserver started at http://127.8.141.194:38135/ using document root <none> and password file <none>
I20260809 23:13:09.606547 7571 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:09.606597 7571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:09.606765 7571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:09.607863 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "1ffcbe5afef1469488ef99d89f5b3b4b"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "244ec0ddcd76e169ecaea83f5000f011"
server_key_iv: "e41cc3ac56dca5e080881958cec10d22"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.608223 7571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "1ffcbe5afef1469488ef99d89f5b3b4b"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "244ec0ddcd76e169ecaea83f5000f011"
server_key_iv: "e41cc3ac56dca5e080881958cec10d22"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.609848 7571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:09.610764 7589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.611054 7571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:09.611186 7571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "1ffcbe5afef1469488ef99d89f5b3b4b"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "244ec0ddcd76e169ecaea83f5000f011"
server_key_iv: "e41cc3ac56dca5e080881958cec10d22"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.611250 7571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:09.628484 7571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:09.628793 7571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:09.628917 7571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:09.629124 7571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:09.629585 7571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:09.629630 7571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.629683 7571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:09.629715 7571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.635928 7571 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:41197
I20260809 23:13:09.635977 7789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:41197 every 8 connection(s)
I20260809 23:13:09.636339 7571 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:13:09.641659 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.637209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:51893 (local address 127.8.141.254:46001)
0809 23:13:09.637258 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.637261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.637509 (+ 248us) server_negotiation.cc:407] Connection header received
0809 23:13:09.637995 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.637996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.638007 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.638073 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:09.638757 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.639258 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.640277 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.640469 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.640527 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.641430 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:09.641431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:09.641431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:09.641435 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:09.641446 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:09.641449 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:09.641489 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.641575 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.641595 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:13:09.641911 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.637144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:46001 (local address 127.8.141.194:51893)
0809 23:13:09.637368 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.637414 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.637914 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.638153 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.638157 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.638166 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:09.638686 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.638690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.639356 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.639356 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.640190 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.640192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.640551 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.640552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.640661 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.641289 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:09.641290 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:09.641319 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:09.641360 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:09.641506 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:09.641511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:09.641514 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.641568 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.641649 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260809 23:13:09.642450 7795 heartbeater.cc:347] Connected to a master server at 127.8.141.254:46001
I20260809 23:13:09.642563 7795 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:09.642779 7795 heartbeater.cc:511] Master 127.8.141.254:46001 requested a full tablet report, sending...
I20260809 23:13:09.643147 7236 ts_manager.cc:194] Registered new tserver with Master: 1ffcbe5afef1469488ef99d89f5b3b4b (127.8.141.194:41197)
I20260809 23:13:09.643532 7236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.194:51893
I20260809 23:13:09.645046 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 7571
I20260809 23:13:09.645174 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:13:09.645403 8759 external_mini_cluster.cc:1428] Setting key 0e64eaf7e75ccb43c68482157a2ada3b
I20260809 23:13:09.646468 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:46001
--builtin_ntp_servers=127.8.141.212:33459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:09.735625 7801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:09.735867 7801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:09.735890 7801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:09.737161 7801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:09.737214 7801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:09.737243 7801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:13:09.738722 7801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:33459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:09.739001 7801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:09.739338 7801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:09.745034 7810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:09.745067 7801 server_base.cc:1034] running on GCE node
W20260809 23:13:09.745034 7817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:09.745040 7809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:09.745671 7801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:09.745941 7801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:09.747115 7801 hybrid_clock.cc:648] HybridClock initialized: now 1786317189747101 us; error 29 us; skew 500 ppm
I20260809 23:13:09.748577 7801 webserver.cc:460] Webserver started at http://127.8.141.195:33299/ using document root <none> and password file <none>
I20260809 23:13:09.748857 7801 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:09.748935 7801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:09.749091 7801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:09.750278 7801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "c13e2ff4a181459ca3bc86c7a0492dda"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "ce6e7bea0d900865b71a864bac040eb0"
server_key_iv: "b0ae19622446026065828f21788ab60e"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.750665 7801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "c13e2ff4a181459ca3bc86c7a0492dda"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "ce6e7bea0d900865b71a864bac040eb0"
server_key_iv: "b0ae19622446026065828f21788ab60e"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.752099 7801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 23:13:09.752977 7827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.753242 7801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:09.753386 7801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "c13e2ff4a181459ca3bc86c7a0492dda"
format_stamp: "Formatted at 2026-08-09 23:13:09 on dist-test-slave-j2vl"
server_key: "ce6e7bea0d900865b71a864bac040eb0"
server_key_iv: "b0ae19622446026065828f21788ab60e"
server_key_version: "encryptionkey@0"
I20260809 23:13:09.753477 7801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:09.763154 7801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:09.763510 7801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:09.763675 7801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:09.763942 7801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:09.764302 7801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:09.764359 7801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.764405 7801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:09.764438 7801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:09.771306 7801 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:43957
I20260809 23:13:09.771401 8005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:43957 every 8 connection(s)
I20260809 23:13:09.771783 7801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:13:09.772409 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 7801
I20260809 23:13:09.772501 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:13:09.772758 8759 external_mini_cluster.cc:1428] Setting key e44451c027ba224f9d30ac61862e249a
I20260809 23:13:09.778034 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.772623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:58155 (local address 127.8.141.254:46001)
0809 23:13:09.772686 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.772689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.773084 (+ 395us) server_negotiation.cc:407] Connection header received
0809 23:13:09.773820 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.773821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.773833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.773902 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:09.774666 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.775197 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.776396 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.776592 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.776657 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.777697 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:09.777699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:09.777699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:09.777704 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:09.777720 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:09.777724 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:09.777770 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.777941 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.777968 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:13:09.778250 8010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.772639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:46001 (local address 127.8.141.195:58155)
0809 23:13:09.772928 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.772974 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.773703 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.773968 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.773972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.773980 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:09.774560 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.774565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.775357 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.775358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.776277 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.776280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.776706 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.776708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.776842 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.777518 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:09.777519 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:09.777551 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:09.777576 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:09.777810 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:09.777817 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:09.777820 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.777915 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.778010 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":149,"threads_started":1}
I20260809 23:13:09.778807 8008 heartbeater.cc:347] Connected to a master server at 127.8.141.254:46001
I20260809 23:13:09.778911 8008 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:09.779074 8008 heartbeater.cc:511] Master 127.8.141.254:46001 requested a full tablet report, sending...
I20260809 23:13:09.779503 7236 ts_manager.cc:194] Registered new tserver with Master: c13e2ff4a181459ca3bc86c7a0492dda (127.8.141.195:43957)
I20260809 23:13:09.779863 7236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.195:58155
I20260809 23:13:09.785319 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:13:09.790220 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.786967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59360 (local address 127.8.141.254:46001)
0809 23:13:09.787054 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.787057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.787189 (+ 132us) server_negotiation.cc:407] Connection header received
0809 23:13:09.787273 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.787273 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.787285 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.787338 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:09.787544 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.788059 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.788840 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.789054 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.789134 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.789900 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:09.789902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:09.789902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:09.789908 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:09.789926 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:09.789930 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:09.790045 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.790113 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.790125 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:13:09.792232 7236 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:13:09.793968 7236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:13:09.801517 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.797114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.8.141.195:43957)
0809 23:13:09.797783 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.797788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.797797 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:09.797838 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.797839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.797854 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.797944 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:09.798070 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.799118 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.800307 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.801013 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.801077 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.801152 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.801361 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.801409 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":92,"threads_started":1}
I20260809 23:13:09.801514 8027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.796698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:43363 (local address 127.0.0.1:41898)
0809 23:13:09.797204 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.797217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.797270 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.797593 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.797596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.797598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:09.797680 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.797685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.799061 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.799062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.800220 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.800222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.801260 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.801261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.801322 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.801325 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.801404 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.801436 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":26,"thread_start_us":42,"threads_started":1}
I20260809 23:13:09.801628 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.796994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.8.141.194:41197)
0809 23:13:09.797453 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.797465 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.797492 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:13:09.797535 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.797537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.797554 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.797640 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:09.797822 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.798678 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.800145 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.800922 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.801000 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.801105 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.801489 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.801535 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":107,"threads_started":1}
I20260809 23:13:09.801824 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.797021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.8.141.193:43363)
0809 23:13:09.797302 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.797307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.797316 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:09.797405 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.797408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.797425 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.797510 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:09.797780 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.798950 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.800311 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.801013 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.801126 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.801224 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.801664 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.801721 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":142,"threads_started":1}
I20260809 23:13:09.802160 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.796639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:41197 (local address 127.0.0.1:41406)
0809 23:13:09.797026 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.797084 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.797165 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.797660 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.797661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.797664 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:09.797714 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.797716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.798771 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.798772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.800051 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.800054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.801145 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.801146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.801213 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.801217 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.801315 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.802088 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":237,"thread_start_us":75,"threads_started":1}
I20260809 23:13:09.802279 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.796874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:43957 (local address 127.0.0.1:56790)
0809 23:13:09.797295 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.797305 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.797362 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.797963 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.797963 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.797966 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:09.798001 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.798002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.799208 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.799209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.800206 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.800209 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.801113 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.801114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.801199 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.801214 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.801296 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.801411 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":17,"thread_start_us":92,"threads_started":1}
I20260809 23:13:09.802547 7459 tablet_service.cc:1467] Processing CreateTablet for tablet f315495b40a6425aa08c025ae14d7b7a (DEFAULT_TABLE table=test-table [id=af3662cf36da4d91866d7387c7111918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:09.802855 7685 tablet_service.cc:1467] Processing CreateTablet for tablet f315495b40a6425aa08c025ae14d7b7a (DEFAULT_TABLE table=test-table [id=af3662cf36da4d91866d7387c7111918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:09.802989 7459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f315495b40a6425aa08c025ae14d7b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:09.802966 7891 tablet_service.cc:1467] Processing CreateTablet for tablet f315495b40a6425aa08c025ae14d7b7a (DEFAULT_TABLE table=test-table [id=af3662cf36da4d91866d7387c7111918]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:09.803182 7685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f315495b40a6425aa08c025ae14d7b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:09.803339 7891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f315495b40a6425aa08c025ae14d7b7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:09.806282 8031 tablet_bootstrap.cc:492] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Bootstrap starting.
I20260809 23:13:09.806710 8034 tablet_bootstrap.cc:492] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Bootstrap starting.
I20260809 23:13:09.807090 8031 tablet_bootstrap.cc:654] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:09.807332 8034 tablet_bootstrap.cc:654] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:09.807499 8031 log.cc:824] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:09.807682 8034 log.cc:824] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:09.808049 8032 tablet_bootstrap.cc:492] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf: Bootstrap starting.
I20260809 23:13:09.808411 8031 tablet_bootstrap.cc:492] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: No bootstrap required, opened a new log
I20260809 23:13:09.808493 8031 ts_tablet_manager.cc:1397] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:09.808595 8034 tablet_bootstrap.cc:492] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: No bootstrap required, opened a new log
I20260809 23:13:09.808669 8034 ts_tablet_manager.cc:1397] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:09.808768 8032 tablet_bootstrap.cc:654] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:09.809089 8032 log.cc:824] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:09.810024 8032 tablet_bootstrap.cc:492] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf: No bootstrap required, opened a new log
I20260809 23:13:09.810101 8032 ts_tablet_manager.cc:1397] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 23:13:09.810839 8031 raft_consensus.cc:348] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.811012 8031 raft_consensus.cc:374] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:09.811058 8031 raft_consensus.cc:729] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c13e2ff4a181459ca3bc86c7a0492dda, State: Initialized, Role: FOLLOWER
I20260809 23:13:09.811195 8031 consensus_queue.cc:260] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.811523 8031 ts_tablet_manager.cc:1428] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:13:09.811650 8008 heartbeater.cc:503] Master 127.8.141.254:46001 was elected leader, sending a full tablet report...
I20260809 23:13:09.811638 8034 raft_consensus.cc:348] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.811852 8034 raft_consensus.cc:374] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:09.811894 8034 raft_consensus.cc:729] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ffcbe5afef1469488ef99d89f5b3b4b, State: Initialized, Role: FOLLOWER
I20260809 23:13:09.812016 8034 consensus_queue.cc:260] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.812331 8034 ts_tablet_manager.cc:1428] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 23:13:09.812352 7795 heartbeater.cc:503] Master 127.8.141.254:46001 was elected leader, sending a full tablet report...
I20260809 23:13:09.813153 8032 raft_consensus.cc:348] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.813324 8032 raft_consensus.cc:374] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:09.813400 8032 raft_consensus.cc:729] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f347b74ada75447e8a8798eb042450bf, State: Initialized, Role: FOLLOWER
I20260809 23:13:09.813530 8032 consensus_queue.cc:260] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.813864 7566 heartbeater.cc:503] Master 127.8.141.254:46001 was elected leader, sending a full tablet report...
I20260809 23:13:09.813885 8032 ts_tablet_manager.cc:1428] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
W20260809 23:13:09.887109 7796 tablet.cc:2369] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:13:09.936050 8044 raft_consensus.cc:482] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:13:09.936303 8044 raft_consensus.cc:504] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.936843 8044 leader_election.cc:288] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c13e2ff4a181459ca3bc86c7a0492dda (127.8.141.195:43957), f347b74ada75447e8a8798eb042450bf (127.8.141.193:43363)
I20260809 23:13:09.941020 8029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.937335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49309 (local address 127.8.141.193:43363)
0809 23:13:09.937421 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.937424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.937436 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:13:09.937492 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.937492 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.937502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.937547 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:09.937838 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.938576 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.939981 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.940543 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.940623 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.940633 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.940861 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.940927 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 23:13:09.941017 7800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.937179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:43363 (local address 127.8.141.194:49309)
0809 23:13:09.937272 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.937287 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.937405 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.937607 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.937608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.937611 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:09.937717 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.937720 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.938675 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.938676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.939882 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.939885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.940655 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.940656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.940766 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.940771 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.940848 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.940900 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 23:13:09.941145 8030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.937335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:51557 (local address 127.8.141.195:43957)
0809 23:13:09.937422 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.937424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.937912 (+ 488us) server_negotiation.cc:407] Connection header received
0809 23:13:09.937958 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.937959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.937970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.938023 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:09.938205 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.938840 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.940104 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.940683 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.940736 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.940741 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.941003 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.941059 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 23:13:09.941430 8050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.937186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:43957 (local address 127.8.141.194:51557)
0809 23:13:09.937798 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:09.937809 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:09.937873 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:09.938054 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:09.938055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:09.938057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:09.938130 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.938131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.938927 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.938928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.940022 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:09.940024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.940782 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.940782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:09.940897 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.940900 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:13:09.941267 (+ 367us) client_negotiation.cc:240] Negotiation successful
0809 23:13:09.941317 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":393,"thread_start_us":95,"threads_started":1}
I20260809 23:13:09.941469 7491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f315495b40a6425aa08c025ae14d7b7a" candidate_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f347b74ada75447e8a8798eb042450bf" is_pre_election: true
I20260809 23:13:09.941668 7491 raft_consensus.cc:2454] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ffcbe5afef1469488ef99d89f5b3b4b in term 0.
I20260809 23:13:09.941938 7594 leader_election.cc:302] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ffcbe5afef1469488ef99d89f5b3b4b, f347b74ada75447e8a8798eb042450bf; no voters:
I20260809 23:13:09.941915 7930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f315495b40a6425aa08c025ae14d7b7a" candidate_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" is_pre_election: true
I20260809 23:13:09.942080 7930 raft_consensus.cc:2454] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ffcbe5afef1469488ef99d89f5b3b4b in term 0.
I20260809 23:13:09.942075 8044 raft_consensus.cc:2781] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:13:09.942137 8044 raft_consensus.cc:482] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:13:09.942194 8044 raft_consensus.cc:3037] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:09.943221 8044 raft_consensus.cc:504] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.943392 8044 leader_election.cc:288] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [CANDIDATE]: Term 1 election: Requested vote from peers c13e2ff4a181459ca3bc86c7a0492dda (127.8.141.195:43957), f347b74ada75447e8a8798eb042450bf (127.8.141.193:43363)
I20260809 23:13:09.943596 7930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f315495b40a6425aa08c025ae14d7b7a" candidate_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c13e2ff4a181459ca3bc86c7a0492dda"
I20260809 23:13:09.943693 7930 raft_consensus.cc:3037] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:09.943701 7491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f315495b40a6425aa08c025ae14d7b7a" candidate_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f347b74ada75447e8a8798eb042450bf"
I20260809 23:13:09.943826 7491 raft_consensus.cc:3037] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:09.944556 7930 raft_consensus.cc:2454] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ffcbe5afef1469488ef99d89f5b3b4b in term 1.
I20260809 23:13:09.944636 7491 raft_consensus.cc:2454] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ffcbe5afef1469488ef99d89f5b3b4b in term 1.
I20260809 23:13:09.944795 7599 leader_election.cc:302] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ffcbe5afef1469488ef99d89f5b3b4b, c13e2ff4a181459ca3bc86c7a0492dda; no voters:
I20260809 23:13:09.944947 8044 raft_consensus.cc:2781] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:09.945182 8044 raft_consensus.cc:686] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 1 LEADER]: Becoming Leader. State: Replica: 1ffcbe5afef1469488ef99d89f5b3b4b, State: Running, Role: LEADER
I20260809 23:13:09.945308 8044 consensus_queue.cc:237] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } }
I20260809 23:13:09.946290 7235 catalog_manager.cc:5495] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ffcbe5afef1469488ef99d89f5b3b4b (127.8.141.194). New cstate: current_term: 1 leader_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1ffcbe5afef1469488ef99d89f5b3b4b" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 41197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 } health_report { overall_health: UNKNOWN } } }
I20260809 23:13:09.984494 8028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:09.982020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.8.141.194:41197)
0809 23:13:09.982096 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:09.982098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:09.982105 (+ 7us) server_negotiation.cc:407] Connection header received
0809 23:13:09.982153 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:09.982154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:09.982176 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:09.982258 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:09.982431 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.982925 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.983691 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:09.983912 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:09.983990 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:09.984118 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:09.984225 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:09.984319 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:09.984390 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 23:13:09.984401 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 23:13:09.986816 7491 raft_consensus.cc:1261] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 1 FOLLOWER]: Refusing update from remote peer 1ffcbe5afef1469488ef99d89f5b3b4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:09.986835 7930 raft_consensus.cc:1261] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 1 FOLLOWER]: Refusing update from remote peer 1ffcbe5afef1469488ef99d89f5b3b4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:09.987283 8052 consensus_queue.cc:1035] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c13e2ff4a181459ca3bc86c7a0492dda" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 43957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:09.987447 8044 consensus_queue.cc:1035] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f347b74ada75447e8a8798eb042450bf" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 43363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:09.991194 8057 mvcc.cc:205] Tried to move back new op lower bound from 7316755210182512640 to 7316755210016718848. Current Snapshot: MvccSnapshot[applied={T|T < 7316755210182512640}]
I20260809 23:13:09.991282 8058 mvcc.cc:205] Tried to move back new op lower bound from 7316755210182512640 to 7316755210016718848. Current Snapshot: MvccSnapshot[applied={T|T < 7316755210182512640}]
I20260809 23:13:09.992751 7235 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:09.992846 7235 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:09.993762 7235 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68b84e266c3f4d3aabdbcdab4e16912d: Sending DeleteTablet for 3 replicas of tablet f315495b40a6425aa08c025ae14d7b7a
I20260809 23:13:09.994235 7891 tablet_service.cc:1514] Processing DeleteTablet for tablet f315495b40a6425aa08c025ae14d7b7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:09 UTC) from {username='slave'} at 127.0.0.1:56790
I20260809 23:13:09.994256 7685 tablet_service.cc:1514] Processing DeleteTablet for tablet f315495b40a6425aa08c025ae14d7b7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:09 UTC) from {username='slave'} at 127.0.0.1:41406
I20260809 23:13:09.994709 8073 tablet_replica.cc:330] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: stopping tablet replica
I20260809 23:13:09.994848 8073 raft_consensus.cc:2229] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:13:09.994987 8073 raft_consensus.cc:2258] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:09.995198 7459 tablet_service.cc:1514] Processing DeleteTablet for tablet f315495b40a6425aa08c025ae14d7b7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:09 UTC) from {username='slave'} at 127.0.0.1:41898
I20260809 23:13:09.995469 8074 tablet_replica.cc:330] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf: stopping tablet replica
I20260809 23:13:09.995635 8074 raft_consensus.cc:2229] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:09.995810 8074 raft_consensus.cc:2258] T f315495b40a6425aa08c025ae14d7b7a P f347b74ada75447e8a8798eb042450bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:09.995738 8073 ts_tablet_manager.cc:1905] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:09.995966 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 7353
I20260809 23:13:10.009245 8073 ts_tablet_manager.cc:1918] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:10.009405 8073 log.cc:1196] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/f315495b40a6425aa08c025ae14d7b7a
I20260809 23:13:10.009692 8073 ts_tablet_manager.cc:1939] T f315495b40a6425aa08c025ae14d7b7a P 1ffcbe5afef1469488ef99d89f5b3b4b: Deleting consensus metadata
I20260809 23:13:10.010329 7212 catalog_manager.cc:4841] TS 1ffcbe5afef1469488ef99d89f5b3b4b (127.8.141.194:41197): tablet f315495b40a6425aa08c025ae14d7b7a (table test-table [id=af3662cf36da4d91866d7387c7111918]) successfully deleted
W20260809 23:13:10.019236 7211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:43363 (error 108)
I20260809 23:13:10.019285 8070 tablet_replica.cc:330] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: stopping tablet replica
I20260809 23:13:10.019455 8070 raft_consensus.cc:2229] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:10.019606 8070 raft_consensus.cc:2258] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:10.019620 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 7571
I20260809 23:13:10.051533 8021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:10.051308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:43363 (local address 127.0.0.1:41912)
0809 23:13:10.051392 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:10.051470 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:43363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 23:13:10.051751 7211 catalog_manager.cc:4584] TS f347b74ada75447e8a8798eb042450bf (127.8.141.193:43363): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f315495b40a6425aa08c025ae14d7b7a: Network error: Client connection negotiation failed: client connection to 127.8.141.193:43363: connect: Connection refused (error 111)
I20260809 23:13:10.056002 8070 ts_tablet_manager.cc:1905] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:10.058511 8070 ts_tablet_manager.cc:1918] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:10.058842 8070 log.cc:1196] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/f315495b40a6425aa08c025ae14d7b7a
I20260809 23:13:10.059201 8070 ts_tablet_manager.cc:1939] T f315495b40a6425aa08c025ae14d7b7a P c13e2ff4a181459ca3bc86c7a0492dda: Deleting consensus metadata
I20260809 23:13:10.059785 7212 catalog_manager.cc:4841] TS c13e2ff4a181459ca3bc86c7a0492dda (127.8.141.195:43957): tablet f315495b40a6425aa08c025ae14d7b7a (table test-table [id=af3662cf36da4d91866d7387c7111918]) successfully deleted
I20260809 23:13:10.061769 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 7801
I20260809 23:13:10.079332 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 7188
2026-08-09T23:13:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (946 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:10 dist-test-slave-j2vl krb5kdc[8083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:10 dist-test-slave-j2vl krb5kdc[8083](info): set up 2 sockets
Aug 09 23:13:10 dist-test-slave-j2vl krb5kdc[8083](info): commencing operation
krb5kdc: starting...
W20260809 23:13:11.418633 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.305s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:11 dist-test-slave-j2vl krb5kdc[8083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:11.556864 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39217
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:39217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:11.643292 8099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:11.643574 8099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:11.643618 8099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:11.644891 8099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:11.644973 8099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:11.645001 8099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:11.645035 8099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:11.645058 8099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:11.646673 8099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:39217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:11.647001 8099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:11.647259 8099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:11.652763 8113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:11.652778 8105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:11.652828 8099 server_base.cc:1034] running on GCE node
W20260809 23:13:11.652778 8107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:11.653301 8099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:11.653710 8099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:11.654902 8099 hybrid_clock.cc:648] HybridClock initialized: now 1786317191654874 us; error 45 us; skew 500 ppm
Aug 09 23:13:11 dist-test-slave-j2vl krb5kdc[8083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317191, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:11.658061 8099 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:11.658550 8099 webserver.cc:460] Webserver started at http://127.8.141.254:38823/ using document root <none> and password file <none>
I20260809 23:13:11.658751 8099 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:11.658793 8099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:11.658916 8099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:11.660009 8099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "5717153f8c754aa7b41125e6e6a34df8"
format_stamp: "Formatted at 2026-08-09 23:13:11 on dist-test-slave-j2vl"
server_key: "710943421bc904413b0450306c6e5a90"
server_key_iv: "9cc3b4e1b0b669d258f544009315bdb5"
server_key_version: "encryptionkey@0"
I20260809 23:13:11.660332 8099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "5717153f8c754aa7b41125e6e6a34df8"
format_stamp: "Formatted at 2026-08-09 23:13:11 on dist-test-slave-j2vl"
server_key: "710943421bc904413b0450306c6e5a90"
server_key_iv: "9cc3b4e1b0b669d258f544009315bdb5"
server_key_version: "encryptionkey@0"
I20260809 23:13:11.662293 8099 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 23:13:11.663198 8133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:11.663458 8099 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:11.663609 8099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "5717153f8c754aa7b41125e6e6a34df8"
format_stamp: "Formatted at 2026-08-09 23:13:11 on dist-test-slave-j2vl"
server_key: "710943421bc904413b0450306c6e5a90"
server_key_iv: "9cc3b4e1b0b669d258f544009315bdb5"
server_key_version: "encryptionkey@0"
I20260809 23:13:11.663707 8099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:11.694824 8099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:11.697943 8099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:11.698252 8099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:11.703267 8099 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:39217
I20260809 23:13:11.703320 8214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:39217 every 8 connection(s)
I20260809 23:13:11.703734 8099 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:11.704771 8215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:11.707262 8215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8: Bootstrap starting.
I20260809 23:13:11.707914 8215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:11.708230 8215 log.cc:824] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:11.709105 8215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8: No bootstrap required, opened a new log
I20260809 23:13:11.711104 8215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5717153f8c754aa7b41125e6e6a34df8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39217 } }
I20260809 23:13:11.711259 8215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:11.711283 8215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5717153f8c754aa7b41125e6e6a34df8, State: Initialized, Role: FOLLOWER
I20260809 23:13:11.711478 8215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5717153f8c754aa7b41125e6e6a34df8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39217 } }
I20260809 23:13:11.711553 8215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:11.711578 8215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:11.711609 8215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:11.712224 8215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5717153f8c754aa7b41125e6e6a34df8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39217 } }
I20260809 23:13:11.712317 8215 leader_election.cc:302] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5717153f8c754aa7b41125e6e6a34df8; no voters:
I20260809 23:13:11.712575 8215 leader_election.cc:288] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:11.712630 8219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:11.712875 8215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:11.712890 8219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [term 1 LEADER]: Becoming Leader. State: Replica: 5717153f8c754aa7b41125e6e6a34df8, State: Running, Role: LEADER
I20260809 23:13:11.713132 8219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5717153f8c754aa7b41125e6e6a34df8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39217 } }
I20260809 23:13:11.713374 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 8099
I20260809 23:13:11.713506 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:11.713747 8759 external_mini_cluster.cc:1428] Setting key 5b23696831e32e6b112e7a1a464470ba
I20260809 23:13:11.713785 8220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5717153f8c754aa7b41125e6e6a34df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5717153f8c754aa7b41125e6e6a34df8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39217 } } }
I20260809 23:13:11.713878 8220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:11.714346 8236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:11.714519 8221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5717153f8c754aa7b41125e6e6a34df8. Latest consensus state: current_term: 1 leader_uuid: "5717153f8c754aa7b41125e6e6a34df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5717153f8c754aa7b41125e6e6a34df8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39217 } } }
I20260809 23:13:11.714797 8221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:11.714985 8236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:11.717451 8236 catalog_manager.cc:1347] Generated new cluster ID: a64b5cdec13e4966a1e88918213482b1
I20260809 23:13:11.717527 8236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:13:11 dist-test-slave-j2vl krb5kdc[8083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:11.723003 8236 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:11.723762 8236 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:11.726675 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:11.714567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.8.141.254:39217)
0809 23:13:11.714778 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:11.714788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:11.714853 (+ 65us) server_negotiation.cc:407] Connection header received
0809 23:13:11.715020 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:11.715027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:11.715092 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:11.715257 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:11.716093 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:11.717471 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:11.718629 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:11.719437 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:11.719567 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:11.722710 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:11.722715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:11.722716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:11.722730 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:11.724809 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:11.725325 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:11.725328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:11.725329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:11.725454 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:11.725684 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:11.725685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:11.725685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:11.725964 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:11.726129 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:11.726280 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:13:11.726433 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":71,"threads_started":1}
I20260809 23:13:11.730515 8236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8: Generated new TSK 0
I20260809 23:13:11.730720 8236 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.141.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
Entry for principal kudu/127.8.141.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab.
I20260809 23:13:11.778786 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39217
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:11.912586 8255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:11.913092 8255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:11.913235 8255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:11.914901 8255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:11.914971 8255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:11.915027 8255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:13:11.916704 8255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.193.keytab
--principal=kudu/127.8.141.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:11.916975 8255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:11.917222 8255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:11.923486 8264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:11.923687 8268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:11.923730 8255 server_base.cc:1034] running on GCE node
W20260809 23:13:11.923545 8261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:11.924027 8255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:11.924309 8255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:11.925556 8255 hybrid_clock.cc:648] HybridClock initialized: now 1786317191925550 us; error 38 us; skew 500 ppm
Aug 09 23:13:11 dist-test-slave-j2vl krb5kdc[8083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317191, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:11.928426 8255 init.cc:377] Logged in from keytab as kudu/127.8.141.193@KRBTEST.COM (short username kudu)
I20260809 23:13:11.928947 8255 webserver.cc:460] Webserver started at http://127.8.141.193:46451/ using document root <none> and password file <none>
I20260809 23:13:11.929193 8255 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:11.929262 8255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:11.929450 8255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:11.930583 8255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "8d6e48f51e434c07b35f6016fada1e13"
format_stamp: "Formatted at 2026-08-09 23:13:11 on dist-test-slave-j2vl"
server_key: "64e3c4417f16912b8bd2aaa31b56a8ab"
server_key_iv: "7638af796c047f0ac4f542bd2c95bc86"
server_key_version: "encryptionkey@0"
I20260809 23:13:11.930966 8255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "8d6e48f51e434c07b35f6016fada1e13"
format_stamp: "Formatted at 2026-08-09 23:13:11 on dist-test-slave-j2vl"
server_key: "64e3c4417f16912b8bd2aaa31b56a8ab"
server_key_iv: "7638af796c047f0ac4f542bd2c95bc86"
server_key_version: "encryptionkey@0"
I20260809 23:13:11.932397 8255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:11.933267 8277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:11.933529 8255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:11.933632 8255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "8d6e48f51e434c07b35f6016fada1e13"
format_stamp: "Formatted at 2026-08-09 23:13:11 on dist-test-slave-j2vl"
server_key: "64e3c4417f16912b8bd2aaa31b56a8ab"
server_key_iv: "7638af796c047f0ac4f542bd2c95bc86"
server_key_version: "encryptionkey@0"
I20260809 23:13:11.933717 8255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:11.942319 8255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:11.944638 8255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:11.944903 8255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:11.945171 8255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:11.945580 8255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:11.945638 8255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:11.945683 8255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:11.945710 8255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:11.952221 8255 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:41167
I20260809 23:13:11.952289 8460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:41167 every 8 connection(s)
I20260809 23:13:11.952674 8255 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:13:11.957552 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 8255
I20260809 23:13:11.957686 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:13:11.958050 8759 external_mini_cluster.cc:1428] Setting key 4ec9ee6b553cbb01a1f88089317c8281
Aug 09 23:13:11 dist-test-slave-j2vl krb5kdc[8083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317191, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.193@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:11.963932 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:11.953774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:48649 (local address 127.8.141.254:39217)
0809 23:13:11.953821 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:11.953826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:11.953985 (+ 159us) server_negotiation.cc:407] Connection header received
0809 23:13:11.954522 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:11.954523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:11.954539 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:11.954608 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:11.955754 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:11.956312 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:11.957483 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:11.957814 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:11.958006 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:11.960653 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:11.960655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:11.960655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:11.960662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:11.962237 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:11.962690 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:11.962691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:11.962692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:11.962758 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:11.963032 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:11.963033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:11.963033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:11.963159 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:11.963250 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:11.963674 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 23:13:11.963820 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:13:11.964196 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:11.953606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39217 (local address 127.8.141.193:48649)
0809 23:13:11.953811 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:11.953862 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:11.954398 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:11.954684 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:11.954687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:11.955406 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:11.955626 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:11.955631 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:11.956433 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:11.956434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:11.957319 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:11.957322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:11.958007 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:11.958008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:11.958183 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:11.958839 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:11.958849 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:11.960531 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:11.962350 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:11.962359 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:11.962362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:11.962606 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:11.962843 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:11.962844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:11.962844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:11.962963 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:11.963273 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:11.963283 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:11.963495 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 23:13:11.963645 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 23:13:11.963858 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260809 23:13:11.964749 8462 heartbeater.cc:347] Connected to a master server at 127.8.141.254:39217
I20260809 23:13:11.964872 8462 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:11.965066 8462 heartbeater.cc:511] Master 127.8.141.254:39217 requested a full tablet report, sending...
I20260809 23:13:11.965696 8154 ts_manager.cc:194] Registered new tserver with Master: 8d6e48f51e434c07b35f6016fada1e13 (127.8.141.193:41167)
I20260809 23:13:11.966383 8154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.193@KRBTEST.COM'} at 127.8.141.193:48649
WARNING: no policy specified for kudu/127.8.141.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
Entry for principal kudu/127.8.141.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab.
I20260809 23:13:11.992496 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39217
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:12.091114 8469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:12.091396 8469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:12.091445 8469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:12.092768 8469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:12.092847 8469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:12.092900 8469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:13:12.094664 8469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.194.keytab
--principal=kudu/127.8.141.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:12.094988 8469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:12.095260 8469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:12.101219 8479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:12.101451 8469 server_base.cc:1034] running on GCE node
W20260809 23:13:12.101312 8475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:12.101217 8477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:12.101795 8469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:12.102048 8469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:12.103272 8469 hybrid_clock.cc:648] HybridClock initialized: now 1786317192103265 us; error 37 us; skew 500 ppm
Aug 09 23:13:12 dist-test-slave-j2vl krb5kdc[8083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:12.106180 8469 init.cc:377] Logged in from keytab as kudu/127.8.141.194@KRBTEST.COM (short username kudu)
I20260809 23:13:12.106796 8469 webserver.cc:460] Webserver started at http://127.8.141.194:40909/ using document root <none> and password file <none>
I20260809 23:13:12.107013 8469 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:12.107059 8469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:12.107156 8469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:12.108212 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "25e8d0b4586a4a52a9b3b4016b54791e"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "386938f0d0cada61f7c5b703a9e99f65"
server_key_iv: "786d8d143d75a45c6049e9c1835f8695"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.108526 8469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "25e8d0b4586a4a52a9b3b4016b54791e"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "386938f0d0cada61f7c5b703a9e99f65"
server_key_iv: "786d8d143d75a45c6049e9c1835f8695"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.110082 8469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:13:12.110942 8489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.111250 8469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:12.111343 8469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "25e8d0b4586a4a52a9b3b4016b54791e"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "386938f0d0cada61f7c5b703a9e99f65"
server_key_iv: "786d8d143d75a45c6049e9c1835f8695"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.111469 8469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:12.127724 8469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:12.130434 8469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:12.130617 8469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:12.130872 8469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:12.131196 8469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:12.131230 8469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.131254 8469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:12.131301 8469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.137605 8469 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:35637
I20260809 23:13:12.137751 8681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:35637 every 8 connection(s)
I20260809 23:13:12.138092 8469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:13:12.139469 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 8469
I20260809 23:13:12.139593 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:13:12.139791 8759 external_mini_cluster.cc:1428] Setting key 124312dafae0f04bddef9d2983c3b54f
Aug 09 23:13:12 dist-test-slave-j2vl krb5kdc[8083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.194@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:12.149945 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.139040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:42267 (local address 127.8.141.254:39217)
0809 23:13:12.139088 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.139090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.139440 (+ 350us) server_negotiation.cc:407] Connection header received
0809 23:13:12.140207 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.140208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.140223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.140297 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:12.141697 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.142222 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.143366 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.143729 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.143836 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.146461 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:12.146463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:12.146463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:12.146474 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:12.148251 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:12.148706 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:12.148707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:12.148708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:12.148776 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:12.149055 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:12.149056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:12.149057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:12.149203 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:12.149287 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.149704 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.149802 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:13:12.150051 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.139073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39217 (local address 127.8.141.194:42267)
0809 23:13:12.139271 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.139324 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:12.140080 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:12.140355 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:12.140359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:12.141238 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:12.141569 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.141576 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.142323 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.142324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.143261 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.143264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.143857 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.143858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.144007 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.144555 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:12.144566 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:12.146346 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:12.148352 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:12.148359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:12.148362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:12.148620 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:12.148853 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:12.148853 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:12.148854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:12.148981 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:12.149306 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:12.149313 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:12.149532 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 23:13:12.149656 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 23:13:12.149813 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.8.141.195@KRBTEST.COM; defaulting to no policy
I20260809 23:13:12.150648 8683 heartbeater.cc:347] Connected to a master server at 127.8.141.254:39217
I20260809 23:13:12.150808 8683 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:12.151046 8683 heartbeater.cc:511] Master 127.8.141.254:39217 requested a full tablet report, sending...
I20260809 23:13:12.151540 8154 ts_manager.cc:194] Registered new tserver with Master: 25e8d0b4586a4a52a9b3b4016b54791e (127.8.141.194:35637)
I20260809 23:13:12.152012 8154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.194@KRBTEST.COM'} at 127.8.141.194:42267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
Entry for principal kudu/127.8.141.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab.
I20260809 23:13:12.174399 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39217
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:36829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:12.265683 8690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:12.265949 8690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:12.265998 8690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:12.267534 8690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:12.267606 8690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:12.267656 8690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:13:12.269447 8690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:36829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.195.keytab
--principal=kudu/127.8.141.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:12.269755 8690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:12.270040 8690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:12.275998 8700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:12.276135 8690 server_base.cc:1034] running on GCE node
W20260809 23:13:12.276005 8698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:12.276005 8703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:12.276547 8690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:12.276791 8690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:12.278016 8690 hybrid_clock.cc:648] HybridClock initialized: now 1786317192278000 us; error 26 us; skew 500 ppm
Aug 09 23:13:12 dist-test-slave-j2vl krb5kdc[8083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:12.280920 8690 init.cc:377] Logged in from keytab as kudu/127.8.141.195@KRBTEST.COM (short username kudu)
I20260809 23:13:12.281548 8690 webserver.cc:460] Webserver started at http://127.8.141.195:33547/ using document root <none> and password file <none>
I20260809 23:13:12.281826 8690 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:12.281906 8690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:12.282071 8690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:12.283260 8690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "84481ae93b0746c6959045f9bf34d055"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "79b31cdf2f2310fbd3df9e246b684704"
server_key_iv: "4a27942854782cbc4a8df344bc5bcff3"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.283732 8690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "84481ae93b0746c6959045f9bf34d055"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "79b31cdf2f2310fbd3df9e246b684704"
server_key_iv: "4a27942854782cbc4a8df344bc5bcff3"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.285230 8690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:12.286211 8714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.286424 8690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:12.286540 8690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "84481ae93b0746c6959045f9bf34d055"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "79b31cdf2f2310fbd3df9e246b684704"
server_key_iv: "4a27942854782cbc4a8df344bc5bcff3"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.286633 8690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:12.303926 8690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:12.306692 8690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:12.306896 8690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:12.307179 8690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:12.307546 8690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:12.307608 8690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.307654 8690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:12.307690 8690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.314651 8690 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:41625
I20260809 23:13:12.314761 8922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:41625 every 8 connection(s)
I20260809 23:13:12.315158 8690 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:13:12.321034 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 8690
I20260809 23:13:12.321177 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:13:12.321406 8759 external_mini_cluster.cc:1428] Setting key 539936f505093ad1f9f5b40e41426d2e
Aug 09 23:13:12 dist-test-slave-j2vl krb5kdc[8083](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317192, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.195@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:12.326176 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.315991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:33633 (local address 127.8.141.254:39217)
0809 23:13:12.316059 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.316062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.316296 (+ 234us) server_negotiation.cc:407] Connection header received
0809 23:13:12.316810 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.316811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.316822 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.316880 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:12.318085 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.318579 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.319854 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.320093 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.320160 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.322996 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:12.322998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:12.322998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:12.323006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:12.324517 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:12.325029 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:12.325031 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:12.325031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:12.325105 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:12.325499 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:12.325500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:12.325500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:12.325625 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:12.325689 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.326031 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.326110 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 23:13:12.326313 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.315925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39217 (local address 127.8.141.195:33633)
0809 23:13:12.316152 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.316203 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:12.316718 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:12.316938 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:12.316941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:12.317679 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:12.317900 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.317905 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.318695 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.318696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.319745 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.319748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.320193 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.320195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.320331 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.320917 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:12.320929 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:12.322872 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:12.324602 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:12.324609 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:12.324612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:12.324928 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:12.325209 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:12.325210 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:12.325211 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:12.325337 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:12.325697 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:12.325704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:12.325863 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 23:13:12.325956 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:13:12.326097 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":111,"threads_started":1}
I20260809 23:13:12.326815 8923 heartbeater.cc:347] Connected to a master server at 127.8.141.254:39217
I20260809 23:13:12.326938 8923 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:12.327155 8923 heartbeater.cc:511] Master 127.8.141.254:39217 requested a full tablet report, sending...
I20260809 23:13:12.327553 8154 ts_manager.cc:194] Registered new tserver with Master: 84481ae93b0746c6959045f9bf34d055 (127.8.141.195:41625)
I20260809 23:13:12.327962 8154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.141.195@KRBTEST.COM'} at 127.8.141.195:33633
I20260809 23:13:12.334046 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:13:12.342289 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.335494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40584 (local address 127.8.141.254:39217)
0809 23:13:12.335558 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.335561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.335670 (+ 109us) server_negotiation.cc:407] Connection header received
0809 23:13:12.335759 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.335760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.335772 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.335823 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:12.336415 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.336908 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.337678 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.337892 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.337978 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.339261 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:12.339262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:12.339263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:12.339270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:12.340712 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:12.341150 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:12.341151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:12.341151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:12.341222 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:12.341477 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:12.341478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:12.341478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:12.341657 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:12.341781 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.342078 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.342180 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:13:12.344447 8154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:13:12.346287 8154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:13:12.353895 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.349456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.8.141.195:41625)
0809 23:13:12.350002 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.350006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.350017 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:13:12.350059 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.350062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.350079 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.350159 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:12.350358 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.351539 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.352819 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.353399 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.353495 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.353567 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.353744 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.353797 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":96,"threads_started":1}
I20260809 23:13:12.353904 8950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.349285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:35637 (local address 127.0.0.1:38302)
0809 23:13:12.349622 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.349636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:12.349700 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:12.350307 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:12.350308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:12.350318 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:12.350355 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.350356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.351296 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.351297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.352574 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.352577 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.353566 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.353567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.353635 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.353640 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:13:12.353681 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 23:13:12.353753 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":278,"threads_started":1}
I20260809 23:13:12.354080 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.349536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.8.141.193:41167)
0809 23:13:12.349903 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.349909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.350270 (+ 361us) server_negotiation.cc:407] Connection header received
0809 23:13:12.350318 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.350320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.350337 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.350403 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:12.350627 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.351709 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.352854 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.353607 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.353669 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.353728 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.353948 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.353995 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":115,"threads_started":1}
I20260809 23:13:12.353880 8949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.349261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:41625 (local address 127.0.0.1:42968)
0809 23:13:12.349544 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.349595 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:12.349670 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:12.350180 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:12.350184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:12.350186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:12.350268 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.350270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.351668 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.351669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.352716 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.352718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.353506 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.353507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.353607 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.353631 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 23:13:12.353711 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:13:12.353760 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260809 23:13:12.354332 8961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.349789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38302 (local address 127.8.141.194:35637)
0809 23:13:12.350151 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.350156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.350168 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:13:12.350211 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.350213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.350230 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.350312 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:12.350404 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.351178 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.352678 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.353458 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.353949 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.354020 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.354044 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.354231 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":193,"threads_started":1}
I20260809 23:13:12.354497 8955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.349304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41167 (local address 127.0.0.1:52626)
0809 23:13:12.350134 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.350147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:12.350206 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:12.350490 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:12.350491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:12.350493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:12.350536 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.350538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.351856 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.351857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.352744 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.352747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.353762 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.353763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.353816 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.353820 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:12.353871 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 23:13:12.353905 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":180,"threads_started":1}
I20260809 23:13:12.355193 8811 tablet_service.cc:1467] Processing CreateTablet for tablet a7d2d611a3314090b372b8512d2d61f6 (DEFAULT_TABLE table=test-table [id=8eabc466adc54074bdb6f5dfa7fb826d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:12.355178 8579 tablet_service.cc:1467] Processing CreateTablet for tablet a7d2d611a3314090b372b8512d2d61f6 (DEFAULT_TABLE table=test-table [id=8eabc466adc54074bdb6f5dfa7fb826d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:12.355188 8352 tablet_service.cc:1467] Processing CreateTablet for tablet a7d2d611a3314090b372b8512d2d61f6 (DEFAULT_TABLE table=test-table [id=8eabc466adc54074bdb6f5dfa7fb826d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:12.355628 8811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d2d611a3314090b372b8512d2d61f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:12.355702 8579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d2d611a3314090b372b8512d2d61f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:12.355702 8352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7d2d611a3314090b372b8512d2d61f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:12.359022 8966 tablet_bootstrap.cc:492] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Bootstrap starting.
I20260809 23:13:12.359476 8969 tablet_bootstrap.cc:492] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Bootstrap starting.
I20260809 23:13:12.359735 8967 tablet_bootstrap.cc:492] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13: Bootstrap starting.
I20260809 23:13:12.359916 8966 tablet_bootstrap.cc:654] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:12.360498 8967 tablet_bootstrap.cc:654] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:12.360607 8966 log.cc:824] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:12.361115 8967 log.cc:824] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:12.361727 8969 tablet_bootstrap.cc:654] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:12.361754 8966 tablet_bootstrap.cc:492] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: No bootstrap required, opened a new log
I20260809 23:13:12.361845 8966 ts_tablet_manager.cc:1397] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:13:12.362095 8967 tablet_bootstrap.cc:492] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13: No bootstrap required, opened a new log
I20260809 23:13:12.362182 8967 ts_tablet_manager.cc:1397] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 23:13:12.362337 8969 log.cc:824] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:12.363332 8969 tablet_bootstrap.cc:492] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: No bootstrap required, opened a new log
I20260809 23:13:12.363420 8969 ts_tablet_manager.cc:1397] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260809 23:13:12.364219 8966 raft_consensus.cc:348] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.364614 8966 raft_consensus.cc:374] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:12.364696 8966 raft_consensus.cc:729] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84481ae93b0746c6959045f9bf34d055, State: Initialized, Role: FOLLOWER
I20260809 23:13:12.364854 8966 consensus_queue.cc:260] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.364776 8967 raft_consensus.cc:348] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.364971 8967 raft_consensus.cc:374] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:12.365046 8967 raft_consensus.cc:729] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d6e48f51e434c07b35f6016fada1e13, State: Initialized, Role: FOLLOWER
I20260809 23:13:12.365382 8966 ts_tablet_manager.cc:1428] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 23:13:12.365482 8967 consensus_queue.cc:260] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.365597 8923 heartbeater.cc:503] Master 127.8.141.254:39217 was elected leader, sending a full tablet report...
I20260809 23:13:12.365883 8967 ts_tablet_manager.cc:1428] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260809 23:13:12.365944 8462 heartbeater.cc:503] Master 127.8.141.254:39217 was elected leader, sending a full tablet report...
I20260809 23:13:12.367136 8969 raft_consensus.cc:348] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.367372 8969 raft_consensus.cc:374] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:12.367422 8969 raft_consensus.cc:729] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e8d0b4586a4a52a9b3b4016b54791e, State: Initialized, Role: FOLLOWER
I20260809 23:13:12.367555 8969 consensus_queue.cc:260] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.367913 8969 ts_tablet_manager.cc:1428] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 23:13:12.367941 8683 heartbeater.cc:503] Master 127.8.141.254:39217 was elected leader, sending a full tablet report...
W20260809 23:13:12.388747 8684 tablet.cc:2369] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 23:13:12.453577 8463 tablet.cc:2369] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:13:12.514957 8974 raft_consensus.cc:482] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:13:12.515108 8974 raft_consensus.cc:504] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.515551 8974 leader_election.cc:288] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 25e8d0b4586a4a52a9b3b4016b54791e (127.8.141.194:35637), 8d6e48f51e434c07b35f6016fada1e13 (127.8.141.193:41167)
I20260809 23:13:12.520267 8957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.516093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:39391 (local address 127.8.141.193:41167)
0809 23:13:12.516153 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.516155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.516360 (+ 205us) server_negotiation.cc:407] Connection header received
0809 23:13:12.516393 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.516395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.516415 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.516462 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:12.516736 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.517472 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.519177 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.519776 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.519869 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.519883 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.520135 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.520189 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 23:13:12.520265 8961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.516102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:59379 (local address 127.8.141.194:35637)
0809 23:13:12.516197 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.516199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.516211 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:13:12.516240 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.516240 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.516250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.516306 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:12.516594 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.517397 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.519177 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.519830 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.519899 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.519909 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.520139 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.520189 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 23:13:12.520290 8926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.515837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:35637 (local address 127.8.141.195:59379)
0809 23:13:12.516072 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.516088 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:12.516155 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:12.516353 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:12.516354 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:12.516356 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:12.516441 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.516459 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.517490 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.517491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.519023 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.519028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.519927 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.519928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.520006 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.520008 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:13:12.520078 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 23:13:12.520147 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":127}
I20260809 23:13:12.520299 8983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.515836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41167 (local address 127.8.141.195:39391)
0809 23:13:12.516265 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.516273 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:12.516347 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:12.516489 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:12.516490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:12.516492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:12.516635 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.516639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.517549 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.517550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.519023 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:12.519028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.519904 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.519905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:12.519999 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.520004 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 23:13:12.520078 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 23:13:12.520145 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":47,"thread_start_us":287,"threads_started":1}
I20260809 23:13:12.520748 8612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2d611a3314090b372b8512d2d61f6" candidate_uuid: "84481ae93b0746c6959045f9bf34d055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" is_pre_election: true
I20260809 23:13:12.520958 8612 raft_consensus.cc:2454] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84481ae93b0746c6959045f9bf34d055 in term 0.
I20260809 23:13:12.520989 8388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2d611a3314090b372b8512d2d61f6" candidate_uuid: "84481ae93b0746c6959045f9bf34d055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d6e48f51e434c07b35f6016fada1e13" is_pre_election: true
I20260809 23:13:12.521142 8388 raft_consensus.cc:2454] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84481ae93b0746c6959045f9bf34d055 in term 0.
I20260809 23:13:12.521320 8723 leader_election.cc:302] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25e8d0b4586a4a52a9b3b4016b54791e, 84481ae93b0746c6959045f9bf34d055; no voters:
I20260809 23:13:12.521505 8974 raft_consensus.cc:2781] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:13:12.521582 8974 raft_consensus.cc:482] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:13:12.521601 8974 raft_consensus.cc:3037] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:12.522791 8974 raft_consensus.cc:504] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.523027 8974 leader_election.cc:288] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [CANDIDATE]: Term 1 election: Requested vote from peers 25e8d0b4586a4a52a9b3b4016b54791e (127.8.141.194:35637), 8d6e48f51e434c07b35f6016fada1e13 (127.8.141.193:41167)
I20260809 23:13:12.523221 8612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2d611a3314090b372b8512d2d61f6" candidate_uuid: "84481ae93b0746c6959045f9bf34d055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e8d0b4586a4a52a9b3b4016b54791e"
I20260809 23:13:12.523221 8388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7d2d611a3314090b372b8512d2d61f6" candidate_uuid: "84481ae93b0746c6959045f9bf34d055" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d6e48f51e434c07b35f6016fada1e13"
I20260809 23:13:12.523312 8612 raft_consensus.cc:3037] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:12.523335 8388 raft_consensus.cc:3037] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:12.524293 8388 raft_consensus.cc:2454] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84481ae93b0746c6959045f9bf34d055 in term 1.
I20260809 23:13:12.524302 8612 raft_consensus.cc:2454] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84481ae93b0746c6959045f9bf34d055 in term 1.
I20260809 23:13:12.524569 8720 leader_election.cc:302] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84481ae93b0746c6959045f9bf34d055, 8d6e48f51e434c07b35f6016fada1e13; no voters:
I20260809 23:13:12.524695 8974 raft_consensus.cc:2781] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:12.524879 8974 raft_consensus.cc:686] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 1 LEADER]: Becoming Leader. State: Replica: 84481ae93b0746c6959045f9bf34d055, State: Running, Role: LEADER
I20260809 23:13:12.524998 8974 consensus_queue.cc:237] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } }
I20260809 23:13:12.526172 8154 catalog_manager.cc:5495] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84481ae93b0746c6959045f9bf34d055 (127.8.141.195). New cstate: current_term: 1 leader_uuid: "84481ae93b0746c6959045f9bf34d055" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84481ae93b0746c6959045f9bf34d055" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 41625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 } health_report { overall_health: UNKNOWN } } }
I20260809 23:13:12.538409 8952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.535696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.8.141.195:41625)
0809 23:13:12.535763 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.535765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.535774 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:12.535828 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.535828 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.535871 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.535965 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:12.536142 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.536705 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.537612 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.537807 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.537874 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.538020 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:12.538129 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:12.538194 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.538225 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.538245 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 23:13:12.540401 8612 raft_consensus.cc:1261] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 1 FOLLOWER]: Refusing update from remote peer 84481ae93b0746c6959045f9bf34d055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:12.540421 8388 raft_consensus.cc:1261] T a7d2d611a3314090b372b8512d2d61f6 P 8d6e48f51e434c07b35f6016fada1e13 [term 1 FOLLOWER]: Refusing update from remote peer 84481ae93b0746c6959045f9bf34d055: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:12.540874 8984 consensus_queue.cc:1035] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e8d0b4586a4a52a9b3b4016b54791e" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 35637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:12.541059 8974 consensus_queue.cc:1035] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d6e48f51e434c07b35f6016fada1e13" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 41167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:12.544174 8990 mvcc.cc:205] Tried to move back new op lower bound from 7316755220642512896 to 7316755220583514112. Current Snapshot: MvccSnapshot[applied={T|T < 7316755220642512896}]
I20260809 23:13:12.546185 8151 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:12.546300 8151 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:12.547402 8151 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5717153f8c754aa7b41125e6e6a34df8: Sending DeleteTablet for 3 replicas of tablet a7d2d611a3314090b372b8512d2d61f6
I20260809 23:13:12.547807 8811 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d2d611a3314090b372b8512d2d61f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:12 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:42968
I20260809 23:13:12.548094 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 8255
I20260809 23:13:12.548120 9009 tablet_replica.cc:330] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: stopping tablet replica
I20260809 23:13:12.552793 9009 raft_consensus.cc:2229] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:13:12.552973 9009 raft_consensus.cc:2258] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:12.548118 8579 tablet_service.cc:1514] Processing DeleteTablet for tablet a7d2d611a3314090b372b8512d2d61f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:12 UTC) from {username='kudu', principal='kudu/127.8.141.254@KRBTEST.COM'} at 127.0.0.1:38302
I20260809 23:13:12.567591 9010 tablet_replica.cc:330] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: stopping tablet replica
I20260809 23:13:12.567809 9010 raft_consensus.cc:2229] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:12.567961 9010 raft_consensus.cc:2258] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:12.571527 9010 ts_tablet_manager.cc:1905] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:12.574327 9010 ts_tablet_manager.cc:1918] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:12.574481 9010 log.cc:1196] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/a7d2d611a3314090b372b8512d2d61f6
I20260809 23:13:12.574810 9010 ts_tablet_manager.cc:1939] T a7d2d611a3314090b372b8512d2d61f6 P 25e8d0b4586a4a52a9b3b4016b54791e: Deleting consensus metadata
I20260809 23:13:12.575702 8137 catalog_manager.cc:4841] TS 25e8d0b4586a4a52a9b3b4016b54791e (127.8.141.194:35637): tablet a7d2d611a3314090b372b8512d2d61f6 (table test-table [id=8eabc466adc54074bdb6f5dfa7fb826d]) successfully deleted
I20260809 23:13:12.576871 9009 ts_tablet_manager.cc:1905] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:12.579401 9009 ts_tablet_manager.cc:1918] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:12.579540 9009 log.cc:1196] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/a7d2d611a3314090b372b8512d2d61f6
I20260809 23:13:12.579881 9009 ts_tablet_manager.cc:1939] T a7d2d611a3314090b372b8512d2d61f6 P 84481ae93b0746c6959045f9bf34d055: Deleting consensus metadata
I20260809 23:13:12.580448 8134 catalog_manager.cc:4841] TS 84481ae93b0746c6959045f9bf34d055 (127.8.141.195:41625): tablet a7d2d611a3314090b372b8512d2d61f6 (table test-table [id=8eabc466adc54074bdb6f5dfa7fb826d]) successfully deleted
W20260809 23:13:12.584563 8135 connection.cc:535] client connection to 127.8.141.193:41167 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 23:13:12.584692 8135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 23:13:12.585389 8955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.585191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:41167 (local address 127.0.0.1:52642)
0809 23:13:12.585271 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.585321 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:41167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 23:13:12.585496 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 8469
W20260809 23:13:12.598448 8135 catalog_manager.cc:4584] TS 8d6e48f51e434c07b35f6016fada1e13 (127.8.141.193:41167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7d2d611a3314090b372b8512d2d61f6: Network error: Client connection negotiation failed: client connection to 127.8.141.193:41167: connect: Connection refused (error 111)
I20260809 23:13:12.616446 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 8690
I20260809 23:13:12.643254 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 8099
2026-08-09T23:13:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2562 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T23:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:12Z Disabled control of system clock
I20260809 23:13:12.686062 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39701
--webserver_interface=127.8.141.254
--webserver_port=0
--builtin_ntp_servers=127.8.141.212:34533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:39701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:12.779430 9022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:12.779824 9022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:12.779862 9022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:12.781152 9022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:12.781206 9022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:12.781219 9022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:12.781232 9022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:12.781245 9022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:12.782801 9022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:39701
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:12.783115 9022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:12.783465 9022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:12.789319 9031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:12.789544 9022 server_base.cc:1034] running on GCE node
W20260809 23:13:12.789386 9036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:12.789319 9034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:12.789990 9022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:12.790377 9022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:12.791595 9022 hybrid_clock.cc:648] HybridClock initialized: now 1786317192791633 us; error 131 us; skew 500 ppm
I20260809 23:13:12.793063 9022 webserver.cc:460] Webserver started at http://127.8.141.254:46255/ using document root <none> and password file <none>
I20260809 23:13:12.793418 9022 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:12.793495 9022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:12.793666 9022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:12.794823 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "6f8c8882ed444a67ae97c42e9cc616dd"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "6c67190f6433fe357ff3bd3e8e1f84c2"
server_key_iv: "68409ef301324657e13335b23c7f87d7"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.795207 9022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "6f8c8882ed444a67ae97c42e9cc616dd"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "6c67190f6433fe357ff3bd3e8e1f84c2"
server_key_iv: "68409ef301324657e13335b23c7f87d7"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.796593 9022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:12.797575 9045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.797802 9022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:12.797946 9022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "6f8c8882ed444a67ae97c42e9cc616dd"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "6c67190f6433fe357ff3bd3e8e1f84c2"
server_key_iv: "68409ef301324657e13335b23c7f87d7"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.798038 9022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:12.808732 9022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:12.809103 9022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:12.809273 9022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:12.814087 9022 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:39701
I20260809 23:13:12.814124 9138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:39701 every 8 connection(s)
I20260809 23:13:12.814553 9022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:12.815635 9141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:12.818081 9141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd: Bootstrap starting.
I20260809 23:13:12.818722 9141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:12.819033 9141 log.cc:824] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:12.819813 9141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd: No bootstrap required, opened a new log
I20260809 23:13:12.821599 9141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39701 } }
I20260809 23:13:12.821743 9141 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:12.821768 9141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f8c8882ed444a67ae97c42e9cc616dd, State: Initialized, Role: FOLLOWER
I20260809 23:13:12.821947 9141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39701 } }
I20260809 23:13:12.822012 9141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:12.822060 9141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:12.822113 9141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:12.822242 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9022
I20260809 23:13:12.822352 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:12.822561 8759 external_mini_cluster.cc:1428] Setting key 464d33254e19d41f55d99714a435aee8
I20260809 23:13:12.822867 9141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39701 } }
I20260809 23:13:12.822989 9141 leader_election.cc:302] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f8c8882ed444a67ae97c42e9cc616dd; no voters:
I20260809 23:13:12.823199 9141 leader_election.cc:288] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:12.823412 9151 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:12.823493 9141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:12.823621 9151 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [term 1 LEADER]: Becoming Leader. State: Replica: 6f8c8882ed444a67ae97c42e9cc616dd, State: Running, Role: LEADER
I20260809 23:13:12.823747 9151 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39701 } }
I20260809 23:13:12.824146 9148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39701 } } }
I20260809 23:13:12.824155 9153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f8c8882ed444a67ae97c42e9cc616dd. Latest consensus state: current_term: 1 leader_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f8c8882ed444a67ae97c42e9cc616dd" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39701 } } }
I20260809 23:13:12.824296 9148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:12.824334 9153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:12.825218 9168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:12.825803 9168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:12.827560 9168 catalog_manager.cc:1347] Generated new cluster ID: 4a44ec88d9904372afc70fcd782545ac
I20260809 23:13:12.827621 9168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:12.829274 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.823146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.8.141.254:39701)
0809 23:13:12.823376 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.823385 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.823420 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:13:12.823542 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:12.823549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:12.823620 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:12.823790 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:12.824032 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.826071 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.827191 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:12.827700 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:12.827857 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:12.828607 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:12.828611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:12.828611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:12.828797 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:12.828829 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:12.828840 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:12.828919 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:12.829003 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 23:13:12.829086 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":84,"threads_started":1}
I20260809 23:13:12.832051 9168 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:12.832804 9168 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:12.841238 9168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd: Generated new TSK 0
I20260809 23:13:12.841536 9168 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:12.843578 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.193:0
--local_ip_for_outbound_sockets=127.8.141.193
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39701
--builtin_ntp_servers=127.8.141.212:34533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 23:13:12.953197 9187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:12.953460 9187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:12.953486 9187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:12.954774 9187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:12.954828 9187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:12.954856 9187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.193
I20260809 23:13:12.956332 9187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.141.193
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.193
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:12.956617 9187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:12.956964 9187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:12.962447 9194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:12.962452 9193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:12.962447 9198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:12.963069 9187 server_base.cc:1034] running on GCE node
I20260809 23:13:12.963317 9187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:12.963624 9187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:12.964794 9187 hybrid_clock.cc:648] HybridClock initialized: now 1786317192964770 us; error 42 us; skew 500 ppm
I20260809 23:13:12.966382 9187 webserver.cc:460] Webserver started at http://127.8.141.193:34267/ using document root <none> and password file <none>
I20260809 23:13:12.966737 9187 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:12.966802 9187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:12.966938 9187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:12.968137 9187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data/instance:
uuid: "318fad5d15c64e0aaccdc565bc3e8f06"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "837b1fe8d992f4fce2bb54a0f483916d"
server_key_iv: "5c1b11572446f4c8f870303813547bf2"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.968524 9187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance:
uuid: "318fad5d15c64e0aaccdc565bc3e8f06"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "837b1fe8d992f4fce2bb54a0f483916d"
server_key_iv: "5c1b11572446f4c8f870303813547bf2"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.970029 9187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 23:13:12.970850 9208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.971016 9187 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:12.971128 9187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/wal
uuid: "318fad5d15c64e0aaccdc565bc3e8f06"
format_stamp: "Formatted at 2026-08-09 23:13:12 on dist-test-slave-j2vl"
server_key: "837b1fe8d992f4fce2bb54a0f483916d"
server_key_iv: "5c1b11572446f4c8f870303813547bf2"
server_key_version: "encryptionkey@0"
I20260809 23:13:12.971211 9187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:12.989727 9187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:12.990083 9187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:12.990243 9187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:12.990527 9187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:12.990935 9187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:12.990974 9187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.991019 9187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:12.991047 9187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:12.998356 9187 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.193:44175
I20260809 23:13:12.998425 9408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.193:44175 every 8 connection(s)
I20260809 23:13:12.998788 9187 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/data/info.pb
I20260809 23:13:13.002102 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9187
I20260809 23:13:13.002194 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-0/wal/instance
I20260809 23:13:13.002435 8759 external_mini_cluster.cc:1428] Setting key a95135c2f3b8ded6c8917e8adea9bb47
I20260809 23:13:13.003579 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.194:0
--local_ip_for_outbound_sockets=127.8.141.194
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39701
--builtin_ntp_servers=127.8.141.212:34533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:13:13.005028 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.999567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:39929 (local address 127.8.141.254:39701)
0809 23:13:12.999654 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:12.999656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:12.999906 (+ 250us) server_negotiation.cc:407] Connection header received
0809 23:13:13.000456 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.000457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.000470 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.000526 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:13.001292 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.001894 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.003246 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.003605 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.003694 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.004666 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:13.004668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:13.004668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:13.004674 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:13.004691 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:13.004696 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:13.004754 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.004849 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.004891 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 23:13:13.005105 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:12.999533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39701 (local address 127.8.141.193:39929)
0809 23:13:12.999774 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:12.999813 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.000351 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.000582 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.000585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.000595 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:13.001209 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.001215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.002020 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.002022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.003145 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.003148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.003725 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.003726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.003910 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.004516 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:13.004517 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:13.004546 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:13.004572 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:13.004771 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:13.004776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:13.004778 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.004832 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.004916 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":131,"threads_started":1}
I20260809 23:13:13.005718 9409 heartbeater.cc:347] Connected to a master server at 127.8.141.254:39701
I20260809 23:13:13.005836 9409 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:13.006047 9409 heartbeater.cc:511] Master 127.8.141.254:39701 requested a full tablet report, sending...
I20260809 23:13:13.006592 9075 ts_manager.cc:194] Registered new tserver with Master: 318fad5d15c64e0aaccdc565bc3e8f06 (127.8.141.193:44175)
I20260809 23:13:13.007577 9075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.193:39929
W20260809 23:13:13.095523 9416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:13.095846 9416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:13.095878 9416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:13.097254 9416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:13.097312 9416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:13.097390 9416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.194
I20260809 23:13:13.098863 9416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.141.194
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.194
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:13.099112 9416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:13.099417 9416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:13:13.105492 9416 server_base.cc:1034] running on GCE node
W20260809 23:13:13.105615 9423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:13.105472 9424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:13.105472 9428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:13.106010 9416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:13.106300 9416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:13.107501 9416 hybrid_clock.cc:648] HybridClock initialized: now 1786317193107480 us; error 42 us; skew 500 ppm
I20260809 23:13:13.109048 9416 webserver.cc:460] Webserver started at http://127.8.141.194:36187/ using document root <none> and password file <none>
I20260809 23:13:13.109484 9416 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:13.109561 9416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:13.109723 9416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:13.110939 9416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data/instance:
uuid: "13ed46548add471887c04705ac37a2ba"
format_stamp: "Formatted at 2026-08-09 23:13:13 on dist-test-slave-j2vl"
server_key: "e40a8f0a84a43b2689f8d0b68264b0a9"
server_key_iv: "0c4442d66aaa0e8ea706ba974b0f66be"
server_key_version: "encryptionkey@0"
I20260809 23:13:13.111310 9416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance:
uuid: "13ed46548add471887c04705ac37a2ba"
format_stamp: "Formatted at 2026-08-09 23:13:13 on dist-test-slave-j2vl"
server_key: "e40a8f0a84a43b2689f8d0b68264b0a9"
server_key_iv: "0c4442d66aaa0e8ea706ba974b0f66be"
server_key_version: "encryptionkey@0"
I20260809 23:13:13.112711 9416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:13.113659 9439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:13.113853 9416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:13.114023 9416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal
uuid: "13ed46548add471887c04705ac37a2ba"
format_stamp: "Formatted at 2026-08-09 23:13:13 on dist-test-slave-j2vl"
server_key: "e40a8f0a84a43b2689f8d0b68264b0a9"
server_key_iv: "0c4442d66aaa0e8ea706ba974b0f66be"
server_key_version: "encryptionkey@0"
I20260809 23:13:13.114105 9416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:13.148123 9416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:13.148548 9416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:13.148715 9416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:13.148985 9416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:13.149399 9416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:13.149452 9416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:13.149497 9416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:13.149534 9416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:13.155539 9416 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.194:37121
I20260809 23:13:13.155584 9660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.194:37121 every 8 connection(s)
I20260809 23:13:13.155925 9416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/data/info.pb
I20260809 23:13:13.161460 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9416
I20260809 23:13:13.161567 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal/instance
I20260809 23:13:13.161727 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.156709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.194:49147 (local address 127.8.141.254:39701)
0809 23:13:13.156800 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.156802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.156987 (+ 185us) server_negotiation.cc:407] Connection header received
0809 23:13:13.157638 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.157640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.157651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.157712 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:13.158500 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.158999 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.160034 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.160279 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.160353 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.161396 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:13.161398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:13.161399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:13.161406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:13.161423 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:13.161428 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:13.161490 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.161606 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.161640 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 23:13:13.161770 8759 external_mini_cluster.cc:1428] Setting key ce20a520ae8e110ca3d2fa9ca84e9a83
I20260809 23:13:13.162046 9663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.156631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39701 (local address 127.8.141.194:49147)
0809 23:13:13.156830 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.156888 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.157515 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.157866 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.157870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.157879 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:13.158419 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.158423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.159095 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.159096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.159928 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.159931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.160411 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.160412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.160543 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.161206 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:13.161207 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:13.161238 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:13.161263 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:13.161495 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:13.161500 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:13.161504 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.161584 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.161698 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":73,"threads_started":1}
I20260809 23:13:13.162575 9661 heartbeater.cc:347] Connected to a master server at 127.8.141.254:39701
I20260809 23:13:13.162691 9661 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:13.162904 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.141.195:0
--local_ip_for_outbound_sockets=127.8.141.195
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39701
--builtin_ntp_servers=127.8.141.212:34533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 23:13:13.162919 9661 heartbeater.cc:511] Master 127.8.141.254:39701 requested a full tablet report, sending...
I20260809 23:13:13.163347 9075 ts_manager.cc:194] Registered new tserver with Master: 13ed46548add471887c04705ac37a2ba (127.8.141.194:37121)
I20260809 23:13:13.163771 9075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.194:49147
W20260809 23:13:13.250010 9667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:13.250252 9667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:13.250276 9667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:13.251665 9667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:13.251732 9667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:13.251771 9667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.141.195
I20260809 23:13:13.253294 9667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.141.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.141.195
--webserver_port=0
--tserver_master_addrs=127.8.141.254:39701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.141.195
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:13.253620 9667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:13.254038 9667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:13.260427 9675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:13.260479 9678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:13.260447 9680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:13.260645 9667 server_base.cc:1034] running on GCE node
I20260809 23:13:13.260967 9667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:13.261270 9667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:13.262486 9667 hybrid_clock.cc:648] HybridClock initialized: now 1786317193262467 us; error 39 us; skew 500 ppm
I20260809 23:13:13.263962 9667 webserver.cc:460] Webserver started at http://127.8.141.195:33713/ using document root <none> and password file <none>
I20260809 23:13:13.264285 9667 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:13.264374 9667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:13.264616 9667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:13.265848 9667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data/instance:
uuid: "74549275b1884178b13109bbb58ab2dd"
format_stamp: "Formatted at 2026-08-09 23:13:13 on dist-test-slave-j2vl"
server_key: "02520f7e10c3c61fa034896717953fd7"
server_key_iv: "ccdaf83a18509f1fda18d5246c0b3ee5"
server_key_version: "encryptionkey@0"
I20260809 23:13:13.266227 9667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance:
uuid: "74549275b1884178b13109bbb58ab2dd"
format_stamp: "Formatted at 2026-08-09 23:13:13 on dist-test-slave-j2vl"
server_key: "02520f7e10c3c61fa034896717953fd7"
server_key_iv: "ccdaf83a18509f1fda18d5246c0b3ee5"
server_key_version: "encryptionkey@0"
I20260809 23:13:13.267616 9667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:13.268399 9689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:13.268687 9667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:13.268790 9667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal
uuid: "74549275b1884178b13109bbb58ab2dd"
format_stamp: "Formatted at 2026-08-09 23:13:13 on dist-test-slave-j2vl"
server_key: "02520f7e10c3c61fa034896717953fd7"
server_key_iv: "ccdaf83a18509f1fda18d5246c0b3ee5"
server_key_version: "encryptionkey@0"
I20260809 23:13:13.268890 9667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:13.280287 9667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:13.280623 9667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:13.280781 9667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:13.281018 9667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 23:13:13.281399 9667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 23:13:13.281450 9667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:13.281493 9667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 23:13:13.281520 9667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:13.287568 9667 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.195:37179
I20260809 23:13:13.287618 9888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.195:37179 every 8 connection(s)
I20260809 23:13:13.287956 9667 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/data/info.pb
I20260809 23:13:13.289276 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9667
I20260809 23:13:13.289439 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal/instance
I20260809 23:13:13.289670 8759 external_mini_cluster.cc:1428] Setting key 287825543ae9ec358a1ea34d3dbf15fd
I20260809 23:13:13.293473 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.288718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.195:59987 (local address 127.8.141.254:39701)
0809 23:13:13.288818 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.288820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.289102 (+ 282us) server_negotiation.cc:407] Connection header received
0809 23:13:13.289709 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.289711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.289732 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.289815 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:13.290689 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.291193 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.292148 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.292331 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.292387 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.293219 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:13.293220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:13.293221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:13.293225 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:13.293237 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:13.293240 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:13.293283 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.293332 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.293398 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 23:13:13.293705 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.288708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:39701 (local address 127.8.141.195:59987)
0809 23:13:13.288940 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.288984 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.289610 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.289888 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.289893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.289904 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:13.290588 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.290595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.291273 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.291274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.292074 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.292077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.292420 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.292421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.292527 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.293098 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 23:13:13.293098 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:13.293125 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 23:13:13.293148 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:13.293283 (+ 135us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:13.293285 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:13.293287 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.293334 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.293461 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":114,"threads_started":1}
I20260809 23:13:13.294252 9889 heartbeater.cc:347] Connected to a master server at 127.8.141.254:39701
I20260809 23:13:13.294366 9889 heartbeater.cc:464] Registering TS with master...
I20260809 23:13:13.294555 9889 heartbeater.cc:511] Master 127.8.141.254:39701 requested a full tablet report, sending...
I20260809 23:13:13.294945 9075 ts_manager.cc:194] Registered new tserver with Master: 74549275b1884178b13109bbb58ab2dd (127.8.141.195:37179)
I20260809 23:13:13.295285 9075 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.141.195:59987
I20260809 23:13:13.301946 8759 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 23:13:13.306675 9150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.303570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.8.141.254:39701)
0809 23:13:13.303649 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.303652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.303743 (+ 91us) server_negotiation.cc:407] Connection header received
0809 23:13:13.303792 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.303793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.303806 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.303868 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:13.304111 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.304650 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.305497 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.305699 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.305769 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.306474 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:13.306475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:13.306475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:13.306480 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:13.306493 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:13.306497 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:13.306543 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.306593 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.306615 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 23:13:13.308238 9075 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 23:13:13.309875 9075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 23:13:13.316159 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.312121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.8.141.194:37121)
0809 23:13:13.312405 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.312411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.312422 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:13:13.312462 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.312464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.312481 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.312568 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:13.312767 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.313730 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.315107 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.315741 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.315841 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.315941 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.316028 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.316067 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260809 23:13:13.316147 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.311994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37121 (local address 127.0.0.1:43158)
0809 23:13:13.312134 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.312177 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.312247 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.312580 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.312583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.312585 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:13.312665 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.312667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.313831 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.313832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.315006 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.315009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.315844 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.315845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.315923 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.315938 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.316013 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.316070 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260809 23:13:13.316641 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.312198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:37179 (local address 127.0.0.1:48612)
0809 23:13:13.312664 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.312675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.312726 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.313090 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.313090 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.313092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:13.313129 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.313130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.314379 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.314380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.315241 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.315243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.316395 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.316396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.316464 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.316468 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.316515 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.316559 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":70,"threads_started":1}
I20260809 23:13:13.316829 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.312501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.8.141.193:44175)
0809 23:13:13.312749 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.312753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.312776 (+ 23us) server_negotiation.cc:407] Connection header received
0809 23:13:13.312827 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.312830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.312845 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.312923 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:13.313079 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.314235 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.315464 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.316158 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.316218 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.316519 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.316695 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.316742 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260809 23:13:13.317036 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.312309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48612 (local address 127.8.141.195:37179)
0809 23:13:13.312853 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.312862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.312870 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:13:13.312919 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.312921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.312936 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.313016 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:13.313202 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.314299 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.315337 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.316054 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.316126 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.316803 (+ 677us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.316833 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.316947 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":97,"threads_started":1}
I20260809 23:13:13.316965 9526 tablet_service.cc:1467] Processing CreateTablet for tablet ec56ecd3ed0748298f3af9bf82ea731d (DEFAULT_TABLE table=test-table [id=920d5aa276ff4d91aefe24c6789f00ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:13.316881 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.312368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:44175 (local address 127.0.0.1:59600)
0809 23:13:13.312682 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.312693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.312733 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.312935 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.312935 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.312937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:13.312993 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.312995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.314327 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.314329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.315381 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.315384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.316553 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.316554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.316605 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.316609 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.316666 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.316808 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":61,"threads_started":1}
I20260809 23:13:13.317413 9526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec56ecd3ed0748298f3af9bf82ea731d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:13.318094 9774 tablet_service.cc:1467] Processing CreateTablet for tablet ec56ecd3ed0748298f3af9bf82ea731d (DEFAULT_TABLE table=test-table [id=920d5aa276ff4d91aefe24c6789f00ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:13.318143 9299 tablet_service.cc:1467] Processing CreateTablet for tablet ec56ecd3ed0748298f3af9bf82ea731d (DEFAULT_TABLE table=test-table [id=920d5aa276ff4d91aefe24c6789f00ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 23:13:13.318465 9299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec56ecd3ed0748298f3af9bf82ea731d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:13.318465 9774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec56ecd3ed0748298f3af9bf82ea731d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:13.320269 9911 tablet_bootstrap.cc:492] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Bootstrap starting.
I20260809 23:13:13.320863 9911 tablet_bootstrap.cc:654] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:13.321085 9911 log.cc:824] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:13.321839 9911 tablet_bootstrap.cc:492] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: No bootstrap required, opened a new log
I20260809 23:13:13.321904 9911 ts_tablet_manager.cc:1397] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:13:13.321915 9917 tablet_bootstrap.cc:492] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06: Bootstrap starting.
I20260809 23:13:13.322623 9917 tablet_bootstrap.cc:654] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:13.322993 9917 log.cc:824] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:13.323336 9915 tablet_bootstrap.cc:492] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Bootstrap starting.
I20260809 23:13:13.323732 9917 tablet_bootstrap.cc:492] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06: No bootstrap required, opened a new log
I20260809 23:13:13.323819 9917 ts_tablet_manager.cc:1397] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 23:13:13.323997 9915 tablet_bootstrap.cc:654] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:13.324337 9915 log.cc:824] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:13.324937 9911 raft_consensus.cc:348] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.325116 9915 tablet_bootstrap.cc:492] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: No bootstrap required, opened a new log
I20260809 23:13:13.325093 9911 raft_consensus.cc:374] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:13.325176 9911 raft_consensus.cc:729] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13ed46548add471887c04705ac37a2ba, State: Initialized, Role: FOLLOWER
I20260809 23:13:13.325186 9915 ts_tablet_manager.cc:1397] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:13.325311 9911 consensus_queue.cc:260] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.325629 9911 ts_tablet_manager.cc:1428] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 23:13:13.325759 9661 heartbeater.cc:503] Master 127.8.141.254:39701 was elected leader, sending a full tablet report...
I20260809 23:13:13.326907 9917 raft_consensus.cc:348] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.327085 9917 raft_consensus.cc:374] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:13.327121 9917 raft_consensus.cc:729] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 318fad5d15c64e0aaccdc565bc3e8f06, State: Initialized, Role: FOLLOWER
I20260809 23:13:13.327227 9917 consensus_queue.cc:260] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.327481 9917 ts_tablet_manager.cc:1428] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 23:13:13.327524 9409 heartbeater.cc:503] Master 127.8.141.254:39701 was elected leader, sending a full tablet report...
I20260809 23:13:13.327785 9915 raft_consensus.cc:348] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.327900 9915 raft_consensus.cc:374] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:13.327951 9915 raft_consensus.cc:729] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74549275b1884178b13109bbb58ab2dd, State: Initialized, Role: FOLLOWER
I20260809 23:13:13.328053 9915 consensus_queue.cc:260] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.328305 9915 ts_tablet_manager.cc:1428] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 23:13:13.328343 9889 heartbeater.cc:503] Master 127.8.141.254:39701 was elected leader, sending a full tablet report...
I20260809 23:13:13.351647 9926 raft_consensus.cc:482] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 23:13:13.351795 9926 raft_consensus.cc:504] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.352221 9926 leader_election.cc:288] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13ed46548add471887c04705ac37a2ba (127.8.141.194:37121), 74549275b1884178b13109bbb58ab2dd (127.8.141.195:37179)
I20260809 23:13:13.355603 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.352413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.194:37121 (local address 127.8.141.193:51085)
0809 23:13:13.352521 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.352536 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.352602 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.352803 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.352804 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.352806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:13.352940 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.352942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.353923 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.353925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.354726 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.354729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.355323 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.355324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.355413 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.355416 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.355477 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.355523 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":31}
I20260809 23:13:13.355602 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.352534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:51085 (local address 127.8.141.194:37121)
0809 23:13:13.352651 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.352653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.352663 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:13:13.352698 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.352699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.352714 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.352776 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:13.353012 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.353806 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.354837 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.355227 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.355309 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.355319 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.355485 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.355524 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 23:13:13.355748 9929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.352548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.195:37179 (local address 127.8.141.193:53321)
0809 23:13:13.352756 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.352767 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:13.352843 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:13.353039 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:13.353040 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:13.353042 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 23:13:13.353104 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.353119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.353955 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.353956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.354750 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:13.354752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.355460 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.355462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:13.355549 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.355552 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 23:13:13.355626 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 23:13:13.355664 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260809 23:13:13.355739 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.352673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.141.193:53321 (local address 127.8.141.195:37179)
0809 23:13:13.352724 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.352726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.352816 (+ 90us) server_negotiation.cc:407] Connection header received
0809 23:13:13.352956 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.352958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.352970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.353022 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 23:13:13.353182 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.353884 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.354985 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.355366 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.355459 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.355467 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.355635 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.355677 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:13:13.356096 9567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec56ecd3ed0748298f3af9bf82ea731d" candidate_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ed46548add471887c04705ac37a2ba" is_pre_election: true
I20260809 23:13:13.356098 9806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec56ecd3ed0748298f3af9bf82ea731d" candidate_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74549275b1884178b13109bbb58ab2dd" is_pre_election: true
I20260809 23:13:13.356261 9567 raft_consensus.cc:2454] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 318fad5d15c64e0aaccdc565bc3e8f06 in term 0.
I20260809 23:13:13.356261 9806 raft_consensus.cc:2454] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 318fad5d15c64e0aaccdc565bc3e8f06 in term 0.
I20260809 23:13:13.356492 9214 leader_election.cc:302] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13ed46548add471887c04705ac37a2ba, 318fad5d15c64e0aaccdc565bc3e8f06; no voters:
I20260809 23:13:13.356680 9926 raft_consensus.cc:2781] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 23:13:13.356761 9926 raft_consensus.cc:482] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 23:13:13.356833 9926 raft_consensus.cc:3037] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:13.357606 9926 raft_consensus.cc:504] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.357750 9926 leader_election.cc:288] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [CANDIDATE]: Term 1 election: Requested vote from peers 13ed46548add471887c04705ac37a2ba (127.8.141.194:37121), 74549275b1884178b13109bbb58ab2dd (127.8.141.195:37179)
I20260809 23:13:13.357985 9806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec56ecd3ed0748298f3af9bf82ea731d" candidate_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74549275b1884178b13109bbb58ab2dd"
I20260809 23:13:13.357992 9567 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec56ecd3ed0748298f3af9bf82ea731d" candidate_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13ed46548add471887c04705ac37a2ba"
I20260809 23:13:13.358085 9567 raft_consensus.cc:3037] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:13.358085 9806 raft_consensus.cc:3037] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:13.359097 9806 raft_consensus.cc:2454] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 318fad5d15c64e0aaccdc565bc3e8f06 in term 1.
I20260809 23:13:13.359097 9567 raft_consensus.cc:2454] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 318fad5d15c64e0aaccdc565bc3e8f06 in term 1.
I20260809 23:13:13.359320 9214 leader_election.cc:302] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 318fad5d15c64e0aaccdc565bc3e8f06, 74549275b1884178b13109bbb58ab2dd; no voters:
I20260809 23:13:13.359434 9926 raft_consensus.cc:2781] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:13.359586 9926 raft_consensus.cc:686] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 1 LEADER]: Becoming Leader. State: Replica: 318fad5d15c64e0aaccdc565bc3e8f06, State: Running, Role: LEADER
I20260809 23:13:13.359684 9926 consensus_queue.cc:237] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } }
I20260809 23:13:13.360702 9075 catalog_manager.cc:5495] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 reported cstate change: term changed from 0 to 1, leader changed from <none> to 318fad5d15c64e0aaccdc565bc3e8f06 (127.8.141.193). New cstate: current_term: 1 leader_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "318fad5d15c64e0aaccdc565bc3e8f06" member_type: VOTER last_known_addr { host: "127.8.141.193" port: 44175 } health_report { overall_health: HEALTHY } } }
I20260809 23:13:13.367527 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.364874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59602 (local address 127.8.141.193:44175)
0809 23:13:13.364931 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:13.364933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:13.364942 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:13.364977 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:13.364978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:13.365005 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:13.365082 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:13.365195 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.365750 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.366689 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:13.366909 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:13.366997 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:13.367167 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:13.367279 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:13.367367 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:13.367390 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 23:13:13.367411 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:13:13.369505 9567 raft_consensus.cc:1261] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 1 FOLLOWER]: Refusing update from remote peer 318fad5d15c64e0aaccdc565bc3e8f06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:13.369594 9806 raft_consensus.cc:1261] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 1 FOLLOWER]: Refusing update from remote peer 318fad5d15c64e0aaccdc565bc3e8f06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:13.369908 9926 consensus_queue.cc:1035] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13ed46548add471887c04705ac37a2ba" member_type: VOTER last_known_addr { host: "127.8.141.194" port: 37121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:13.370072 9930 consensus_queue.cc:1035] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74549275b1884178b13109bbb58ab2dd" member_type: VOTER last_known_addr { host: "127.8.141.195" port: 37179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:13.375449 9074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:13.375571 9074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 23:13:13.376575 9074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f8c8882ed444a67ae97c42e9cc616dd: Sending DeleteTablet for 3 replicas of tablet ec56ecd3ed0748298f3af9bf82ea731d
I20260809 23:13:13.377206 9774 tablet_service.cc:1514] Processing DeleteTablet for tablet ec56ecd3ed0748298f3af9bf82ea731d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:13 UTC) from {username='slave'} at 127.0.0.1:48612
I20260809 23:13:13.377156 9526 tablet_service.cc:1514] Processing DeleteTablet for tablet ec56ecd3ed0748298f3af9bf82ea731d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:13 UTC) from {username='slave'} at 127.0.0.1:43158
I20260809 23:13:13.377411 9299 tablet_service.cc:1514] Processing DeleteTablet for tablet ec56ecd3ed0748298f3af9bf82ea731d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 23:13:13 UTC) from {username='slave'} at 127.0.0.1:59600
I20260809 23:13:13.377660 9959 tablet_replica.cc:330] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06: stopping tablet replica
I20260809 23:13:13.377775 9958 tablet_replica.cc:330] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: stopping tablet replica
I20260809 23:13:13.377790 9959 raft_consensus.cc:2229] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 1 LEADER]: Raft consensus shutting down.
I20260809 23:13:13.377897 9959 raft_consensus.cc:2258] T ec56ecd3ed0748298f3af9bf82ea731d P 318fad5d15c64e0aaccdc565bc3e8f06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:13.378006 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9187
I20260809 23:13:13.378018 9958 raft_consensus.cc:2229] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:13.378140 9958 raft_consensus.cc:2258] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:13.381608 9958 ts_tablet_manager.cc:1905] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:13.384917 9958 ts_tablet_manager.cc:1918] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:13.385010 9958 log.cc:1196] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-1/wal/wals/ec56ecd3ed0748298f3af9bf82ea731d
I20260809 23:13:13.385257 9958 ts_tablet_manager.cc:1939] T ec56ecd3ed0748298f3af9bf82ea731d P 13ed46548add471887c04705ac37a2ba: Deleting consensus metadata
I20260809 23:13:13.385771 9055 catalog_manager.cc:4841] TS 13ed46548add471887c04705ac37a2ba (127.8.141.194:37121): tablet ec56ecd3ed0748298f3af9bf82ea731d (table test-table [id=920d5aa276ff4d91aefe24c6789f00ac]) successfully deleted
I20260809 23:13:13.395377 9957 tablet_replica.cc:330] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: stopping tablet replica
I20260809 23:13:13.395581 9957 raft_consensus.cc:2229] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 23:13:13.395740 9957 raft_consensus.cc:2258] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 23:13:13.396245 9957 ts_tablet_manager.cc:1905] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 23:13:13.398006 9957 ts_tablet_manager.cc:1918] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 23:13:13.398106 9957 log.cc:1196] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Deleting WAL directory at /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786316926160006-8759-0/minicluster-data/ts-2/wal/wals/ec56ecd3ed0748298f3af9bf82ea731d
I20260809 23:13:13.398379 9957 ts_tablet_manager.cc:1939] T ec56ecd3ed0748298f3af9bf82ea731d P 74549275b1884178b13109bbb58ab2dd: Deleting consensus metadata
I20260809 23:13:13.398926 9055 catalog_manager.cc:4841] TS 74549275b1884178b13109bbb58ab2dd (127.8.141.195:37179): tablet ec56ecd3ed0748298f3af9bf82ea731d (table test-table [id=920d5aa276ff4d91aefe24c6789f00ac]) successfully deleted
W20260809 23:13:13.400367 9055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.141.193:44175 (error 108)
I20260809 23:13:13.400604 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9416
I20260809 23:13:13.401289 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:13.400792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.193:44175 (local address 127.0.0.1:59612)
0809 23:13:13.401220 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:13.401258 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.193:44175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404}
W20260809 23:13:13.409588 9055 catalog_manager.cc:4584] TS 318fad5d15c64e0aaccdc565bc3e8f06 (127.8.141.193:44175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec56ecd3ed0748298f3af9bf82ea731d: Network error: Client connection negotiation failed: client connection to 127.8.141.193:44175: connect: Connection refused (error 111)
I20260809 23:13:13.417583 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9667
I20260809 23:13:13.442385 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9022
2026-08-09T23:13:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (796 ms)
[----------] 6 tests from EncryptionPolicyTest (9964 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:13 dist-test-slave-j2vl krb5kdc[9971](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:13 dist-test-slave-j2vl krb5kdc[9971](info): set up 2 sockets
Aug 09 23:13:13 dist-test-slave-j2vl krb5kdc[9971](info): commencing operation
krb5kdc: starting...
W20260809 23:13:14.770618 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:14 dist-test-slave-j2vl krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:14.908677 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:14.999787 9994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:15.000072 9994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:15.000113 9994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:15.001331 9994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:15.001437 9994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:15.001462 9994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:15.001487 9994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:15.001514 9994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:15.003173 9994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37983
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:15.003515 9994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:15.003773 9994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:15.008945 10003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:15.008913 10008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:15.008913 10001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:15.009187 9994 server_base.cc:1034] running on GCE node
I20260809 23:13:15.009531 9994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:15.009848 9994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:15.011024 9994 hybrid_clock.cc:648] HybridClock initialized: now 1786317195011012 us; error 45 us; skew 500 ppm
Aug 09 23:13:15 dist-test-slave-j2vl krb5kdc[9971](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317195, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:15.016552 9994 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:15.017098 9994 webserver.cc:460] Webserver started at http://127.0.0.1:35501/ using document root <none> and password file <none>
I20260809 23:13:15.017434 9994 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:15.017509 9994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:15.017670 9994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:15.018788 9994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "c35f712f344d450d8434b99028d38eeb"
format_stamp: "Formatted at 2026-08-09 23:13:15 on dist-test-slave-j2vl"
server_key: "7190ab05d0b31a0d5fb2cea4398eabb6"
server_key_iv: "e09f86e3150d6c676ca20a35de11b6d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:15.019170 9994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "c35f712f344d450d8434b99028d38eeb"
format_stamp: "Formatted at 2026-08-09 23:13:15 on dist-test-slave-j2vl"
server_key: "7190ab05d0b31a0d5fb2cea4398eabb6"
server_key_iv: "e09f86e3150d6c676ca20a35de11b6d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:15.020818 9994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:15.021797 10018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:15.022054 9994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:15.022178 9994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "c35f712f344d450d8434b99028d38eeb"
format_stamp: "Formatted at 2026-08-09 23:13:15 on dist-test-slave-j2vl"
server_key: "7190ab05d0b31a0d5fb2cea4398eabb6"
server_key_iv: "e09f86e3150d6c676ca20a35de11b6d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:15.022267 9994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:15.039934 9994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:15.042276 9994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:15.042461 9994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:15.046962 9994 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37983
I20260809 23:13:15.046994 10109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37983 every 8 connection(s)
I20260809 23:13:15.047401 9994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:15.048461 10111 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:15.050938 10111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb: Bootstrap starting.
I20260809 23:13:15.051589 10111 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:15.051916 10111 log.cc:824] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:15.052712 10111 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb: No bootstrap required, opened a new log
I20260809 23:13:15.054697 10111 raft_consensus.cc:348] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35f712f344d450d8434b99028d38eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37983 } }
I20260809 23:13:15.054868 10111 raft_consensus.cc:374] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:15.054893 10111 raft_consensus.cc:729] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c35f712f344d450d8434b99028d38eeb, State: Initialized, Role: FOLLOWER
I20260809 23:13:15.055052 10111 consensus_queue.cc:260] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35f712f344d450d8434b99028d38eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37983 } }
I20260809 23:13:15.055138 10111 raft_consensus.cc:388] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:15.055182 10111 raft_consensus.cc:482] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:15.055258 10111 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:15.055449 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 9994
I20260809 23:13:15.055575 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:15.055773 8759 external_mini_cluster.cc:1428] Setting key 5bba812ffa9930277598e48e13a4819c
I20260809 23:13:15.056018 10111 raft_consensus.cc:504] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35f712f344d450d8434b99028d38eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37983 } }
I20260809 23:13:15.056146 10111 leader_election.cc:302] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c35f712f344d450d8434b99028d38eeb; no voters:
I20260809 23:13:15.056456 10111 leader_election.cc:288] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:15.056565 10114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:15.056823 10114 raft_consensus.cc:686] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [term 1 LEADER]: Becoming Leader. State: Replica: c35f712f344d450d8434b99028d38eeb, State: Running, Role: LEADER
I20260809 23:13:15.056878 10111 sys_catalog.cc:564] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:15.057104 10114 consensus_queue.cc:237] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35f712f344d450d8434b99028d38eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37983 } }
I20260809 23:13:15.058403 10120 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c35f712f344d450d8434b99028d38eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35f712f344d450d8434b99028d38eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37983 } } }
I20260809 23:13:15.058531 10120 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:15.058987 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c35f712f344d450d8434b99028d38eeb. Latest consensus state: current_term: 1 leader_uuid: "c35f712f344d450d8434b99028d38eeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35f712f344d450d8434b99028d38eeb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37983 } } }
I20260809 23:13:15.058811 10142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:15.059567 10142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:15.060240 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:15.061196 10142 catalog_manager.cc:1347] Generated new cluster ID: b75ae615e0ef45b58fe20bc122272541
I20260809 23:13:15.061282 10142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 23:13:15 dist-test-slave-j2vl krb5kdc[9971](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:13:15.067633 10117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:15.056639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52495 (local address 127.0.0.1:37983)
0809 23:13:15.056864 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:15.056875 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:15.056973 (+ 98us) server_negotiation.cc:407] Connection header received
0809 23:13:15.057130 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:15.057138 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:15.057208 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:15.057450 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:15.058206 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:15.059847 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:15.060767 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:15.061420 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:15.061551 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:15.064135 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:15.064139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:15.064140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:15.064154 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:15.065971 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:15.066391 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:15.066394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:15.066394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:15.066510 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:15.066753 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:15.066754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:15.066755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:15.067023 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:15.067210 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:15.067324 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 23:13:15.067435 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
I20260809 23:13:15.069211 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:15.077453 10142 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:15.078008 10142 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:15.081166 10117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:15.073758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48539 (local address 127.0.0.1:37983)
0809 23:13:15.073812 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:15.073815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:15.074245 (+ 430us) server_negotiation.cc:407] Connection header received
0809 23:13:15.074321 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:15.074322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:15.074338 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:15.074419 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:15.075034 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:15.075568 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:15.076274 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:15.076504 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:15.076581 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:15.078237 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:15.078238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:15.078238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:15.078245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:15.079846 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:15.080282 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:15.080283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:15.080283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:15.080343 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:15.080571 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:15.080572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:15.080572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:15.080694 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:15.080796 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:15.081010 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 23:13:15.081075 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:13:15.088244 10142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c35f712f344d450d8434b99028d38eeb: Generated new TSK 0
I20260809 23:13:15.088477 10142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:15.091648 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 9994
W20260809 23:13:15.106235 9968 connection.cc:535] client connection to 127.0.0.1:37983 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37983): Connection reset by peer (error 104)
2026-08-09T23:13:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:15 dist-test-slave-j2vl krb5kdc[10164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:15 dist-test-slave-j2vl krb5kdc[10164](info): set up 2 sockets
Aug 09 23:13:15 dist-test-slave-j2vl krb5kdc[10164](info): commencing operation
krb5kdc: starting...
W20260809 23:13:16.422366 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:16 dist-test-slave-j2vl krb5kdc[10164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:16.561087 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45121
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:16.649390 10184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:16.649621 10184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:16.649649 10184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:16.650800 10184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:16.650841 10184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:16.650856 10184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:16.650868 10184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:16.650880 10184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:16.652375 10184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45121
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:16.652618 10184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:16.652827 10184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:16.658085 10189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:16.658123 10194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:16.658071 10192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:16.658670 10184 server_base.cc:1034] running on GCE node
I20260809 23:13:16.658987 10184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:16.659332 10184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:16.660503 10184 hybrid_clock.cc:648] HybridClock initialized: now 1786317196660496 us; error 50 us; skew 500 ppm
Aug 09 23:13:16 dist-test-slave-j2vl krb5kdc[10164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:16.663113 10184 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:16.663599 10184 webserver.cc:460] Webserver started at http://127.0.0.1:40163/ using document root <none> and password file <none>
I20260809 23:13:16.663822 10184 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:16.663864 10184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:16.663959 10184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:16.664980 10184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "1afc56e1fcac465fbc7917be213d128d"
format_stamp: "Formatted at 2026-08-09 23:13:16 on dist-test-slave-j2vl"
server_key: "579b9b5b0b0b63428f9d234be60d8e49"
server_key_iv: "04c768c365775cd1d8fb4bef67e90553"
server_key_version: "encryptionkey@0"
I20260809 23:13:16.665288 10184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "1afc56e1fcac465fbc7917be213d128d"
format_stamp: "Formatted at 2026-08-09 23:13:16 on dist-test-slave-j2vl"
server_key: "579b9b5b0b0b63428f9d234be60d8e49"
server_key_iv: "04c768c365775cd1d8fb4bef67e90553"
server_key_version: "encryptionkey@0"
I20260809 23:13:16.666718 10184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:16.667557 10204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:16.667737 10184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:16.667840 10184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "1afc56e1fcac465fbc7917be213d128d"
format_stamp: "Formatted at 2026-08-09 23:13:16 on dist-test-slave-j2vl"
server_key: "579b9b5b0b0b63428f9d234be60d8e49"
server_key_iv: "04c768c365775cd1d8fb4bef67e90553"
server_key_version: "encryptionkey@0"
I20260809 23:13:16.667900 10184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:16.680289 10184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:16.682528 10184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:16.682648 10184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:16.686625 10184 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45121
I20260809 23:13:16.686710 10279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45121 every 8 connection(s)
I20260809 23:13:16.687041 10184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:16.687577 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10184
I20260809 23:13:16.687676 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:16.687904 8759 external_mini_cluster.cc:1428] Setting key 7db1b17121214968a5b70961cc27a463
I20260809 23:13:16.688213 10280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:16.691555 10280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d: Bootstrap starting.
I20260809 23:13:16.692262 10280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:16.692612 10280 log.cc:824] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:16.693504 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:16.688388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50351 (local address 127.0.0.1:45121)
0809 23:13:16.688612 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:16.688620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:16.688647 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:13:16.689106 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:16.689112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:16.689157 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:16.689274 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:16.689539 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:16.690411 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:16.691505 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:16.691966 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:16.692053 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:16.692775 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:16.692780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:16.692780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:16.692963 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:16.692993 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:16.693003 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:16.693070 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:16.693210 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 23:13:16.693250 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":113,"threads_started":1}
I20260809 23:13:16.693516 10280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d: No bootstrap required, opened a new log
I20260809 23:13:16.694602 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:16.695590 10280 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1afc56e1fcac465fbc7917be213d128d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260809 23:13:16.695734 10280 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:16.695775 10280 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1afc56e1fcac465fbc7917be213d128d, State: Initialized, Role: FOLLOWER
I20260809 23:13:16.695897 10280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1afc56e1fcac465fbc7917be213d128d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260809 23:13:16.695981 10280 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:16.696059 10280 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:16.696110 10280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:16.696702 10280 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1afc56e1fcac465fbc7917be213d128d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260809 23:13:16.696822 10280 leader_election.cc:302] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1afc56e1fcac465fbc7917be213d128d; no voters:
I20260809 23:13:16.697026 10280 leader_election.cc:288] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:16.697058 10296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:16.697166 10296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [term 1 LEADER]: Becoming Leader. State: Replica: 1afc56e1fcac465fbc7917be213d128d, State: Running, Role: LEADER
I20260809 23:13:16.697297 10280 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:16.697305 10296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1afc56e1fcac465fbc7917be213d128d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } }
I20260809 23:13:16.699918 10297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1afc56e1fcac465fbc7917be213d128d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1afc56e1fcac465fbc7917be213d128d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } } }
I20260809 23:13:16.700038 10297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:16.700582 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:16.696006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52347 (local address 127.0.0.1:45121)
0809 23:13:16.696050 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:16.696051 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:16.696159 (+ 108us) server_negotiation.cc:407] Connection header received
0809 23:13:16.696230 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:16.696231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:16.696241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:16.696295 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:16.696412 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:16.696939 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:16.697915 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:16.698170 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:16.698205 (+ 35us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:16.700284 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:16.700285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:16.700286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:16.700291 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:16.700306 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:16.700310 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:16.700341 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:16.700478 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 23:13:16.700488 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:13:16.700727 10299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1afc56e1fcac465fbc7917be213d128d. Latest consensus state: current_term: 1 leader_uuid: "1afc56e1fcac465fbc7917be213d128d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1afc56e1fcac465fbc7917be213d128d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45121 } } }
I20260809 23:13:16.700876 10299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d [sys.catalog]: This master's current role is: LEADER
W20260809 23:13:16.701464 10322 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:16.701557 10322 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:16.701794 10310 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:16.701993 10310 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:16.703719 10310 catalog_manager.cc:1347] Generated new cluster ID: 2c8172dd828d4d1482d990ebca09cca8
I20260809 23:13:16.703790 10310 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:16.718007 10310 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:16.718714 10310 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:16.724344 10310 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1afc56e1fcac465fbc7917be213d128d: Generated new TSK 0
I20260809 23:13:16.724567 10310 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:16.727022 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10184
W20260809 23:13:16.740038 10161 connection.cc:535] client connection to 127.0.0.1:45121 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45121): Connection reset by peer (error 104)
2026-08-09T23:13:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1633 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:16 dist-test-slave-j2vl krb5kdc[10336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:16 dist-test-slave-j2vl krb5kdc[10336](info): set up 2 sockets
Aug 09 23:13:16 dist-test-slave-j2vl krb5kdc[10336](info): commencing operation
krb5kdc: starting...
W20260809 23:13:18.046450 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:18 dist-test-slave-j2vl krb5kdc[10336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:18.181725 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:18.271687 10356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:18.271943 10356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:18.271982 10356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:18.273655 10356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:18.273715 10356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:18.273736 10356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:18.273761 10356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:18.273794 10356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:18.275420 10356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:18.275686 10356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:18.275982 10356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:18.282342 10365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:18.282366 10366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:18.282354 10369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:18.282536 10356 server_base.cc:1034] running on GCE node
I20260809 23:13:18.282862 10356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:18.283243 10356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:18.284446 10356 hybrid_clock.cc:648] HybridClock initialized: now 1786317198284423 us; error 32 us; skew 500 ppm
Aug 09 23:13:18 dist-test-slave-j2vl krb5kdc[10336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:18.287496 10356 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:18.287976 10356 webserver.cc:460] Webserver started at http://127.0.0.1:41923/ using document root <none> and password file <none>
I20260809 23:13:18.288255 10356 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:18.288326 10356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:18.288475 10356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:18.289633 10356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "ecd199bf7b44481ab5e6e42c3935a2ec"
format_stamp: "Formatted at 2026-08-09 23:13:18 on dist-test-slave-j2vl"
server_key: "dfa4126968c9b847d89cf37ff6f84aab"
server_key_iv: "5f0e37d1dbc5404a0626d171f0c2e5fa"
server_key_version: "encryptionkey@0"
I20260809 23:13:18.289992 10356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "ecd199bf7b44481ab5e6e42c3935a2ec"
format_stamp: "Formatted at 2026-08-09 23:13:18 on dist-test-slave-j2vl"
server_key: "dfa4126968c9b847d89cf37ff6f84aab"
server_key_iv: "5f0e37d1dbc5404a0626d171f0c2e5fa"
server_key_version: "encryptionkey@0"
I20260809 23:13:18.291430 10356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:18.292263 10379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:18.292470 10356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:18.292578 10356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "ecd199bf7b44481ab5e6e42c3935a2ec"
format_stamp: "Formatted at 2026-08-09 23:13:18 on dist-test-slave-j2vl"
server_key: "dfa4126968c9b847d89cf37ff6f84aab"
server_key_iv: "5f0e37d1dbc5404a0626d171f0c2e5fa"
server_key_version: "encryptionkey@0"
I20260809 23:13:18.292654 10356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:18.296129 10356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:18.298164 10356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:18.298326 10356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:18.302640 10356 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45175
I20260809 23:13:18.302706 10484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45175 every 8 connection(s)
I20260809 23:13:18.303071 10356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:18.304159 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:18.306900 10486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec: Bootstrap starting.
I20260809 23:13:18.307519 10486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:18.307806 10486 log.cc:824] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:18.307843 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10356
I20260809 23:13:18.307991 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:18.308204 8759 external_mini_cluster.cc:1428] Setting key f58e384342e3926df2b6d955dcd26081
I20260809 23:13:18.308713 10486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec: No bootstrap required, opened a new log
I20260809 23:13:18.310961 10486 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45175 } }
I20260809 23:13:18.311126 10486 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:18.311178 10486 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecd199bf7b44481ab5e6e42c3935a2ec, State: Initialized, Role: FOLLOWER
I20260809 23:13:18.311331 10486 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45175 } }
I20260809 23:13:18.311448 10486 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:18.311494 10486 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:18.311544 10486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:18.311693 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:18.308793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53555 (local address 127.0.0.1:45175)
0809 23:13:18.309002 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:18.309013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:18.309052 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:13:18.310089 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:18.310098 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:18.310127 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:18.310286 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:18.311021 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:18.311024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:18.311025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:18.311028 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:18.311064 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:18.311075 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:18.311152 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:18.311265 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 23:13:18.311313 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260809 23:13:18.312282 10486 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45175 } }
I20260809 23:13:18.312397 10486 leader_election.cc:302] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecd199bf7b44481ab5e6e42c3935a2ec; no voters:
I20260809 23:13:18.312572 10486 leader_election.cc:288] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:18.312646 10500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:18.312667 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:18.312816 10500 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [term 1 LEADER]: Becoming Leader. State: Replica: ecd199bf7b44481ab5e6e42c3935a2ec, State: Running, Role: LEADER
I20260809 23:13:18.312847 10486 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:18.312963 10500 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45175 } }
I20260809 23:13:18.314213 10502 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecd199bf7b44481ab5e6e42c3935a2ec. Latest consensus state: current_term: 1 leader_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45175 } } }
I20260809 23:13:18.314317 10502 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:18.314355 10501 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecd199bf7b44481ab5e6e42c3935a2ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45175 } } }
I20260809 23:13:18.314425 10501 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec [sys.catalog]: This master's current role is: LEADER
W20260809 23:13:18.314750 10530 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:18.314872 10530 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:18.315011 10534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:18.315227 10534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:18.316758 10499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:18.314401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60359 (local address 127.0.0.1:45175)
0809 23:13:18.314440 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:18.314442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:18.314755 (+ 313us) server_negotiation.cc:407] Connection header received
0809 23:13:18.314926 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:18.314927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:18.314933 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:18.314963 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:18.316462 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:18.316463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:18.316463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:18.316468 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:18.316483 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:18.316487 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:18.316532 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:18.316675 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 23:13:18.316689 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 23:13:18.316993 10534 catalog_manager.cc:1347] Generated new cluster ID: dc8798af0c054ee5a60a4f86d25697b7
I20260809 23:13:18.317041 10534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:18.340349 10534 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:18.340581 10534 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:18.348207 10534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecd199bf7b44481ab5e6e42c3935a2ec: Generated new TSK 0
I20260809 23:13:18.348448 10534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:18.353092 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10356
2026-08-09T23:13:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1622 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:18 dist-test-slave-j2vl krb5kdc[10542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:18 dist-test-slave-j2vl krb5kdc[10542](info): set up 2 sockets
Aug 09 23:13:18 dist-test-slave-j2vl krb5kdc[10542](info): commencing operation
krb5kdc: starting...
W20260809 23:13:19.660010 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:19 dist-test-slave-j2vl krb5kdc[10542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:19.796885 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:19.883411 10569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:19.883697 10569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:19.883742 10569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:19.885006 10569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:19.885089 10569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:19.885114 10569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:19.885138 10569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:19.885165 10569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:19.886797 10569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:19.887284 10569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:19.887542 10569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:19.892684 10582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:19.892663 10580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:19.892666 10593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:19.892978 10569 server_base.cc:1034] running on GCE node
I20260809 23:13:19.893236 10569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:19.893577 10569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:19.894752 10569 hybrid_clock.cc:648] HybridClock initialized: now 1786317199894728 us; error 54 us; skew 500 ppm
Aug 09 23:13:19 dist-test-slave-j2vl krb5kdc[10542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317199, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:19.897459 10569 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:19.897933 10569 webserver.cc:460] Webserver started at http://127.0.0.1:40697/ using document root <none> and password file <none>
I20260809 23:13:19.898168 10569 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:19.898216 10569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:19.898365 10569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:19.899441 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "e3fe17f48c5f4077a625b1de6decdce0"
format_stamp: "Formatted at 2026-08-09 23:13:19 on dist-test-slave-j2vl"
server_key: "6c98a8e6b052a58220be747eef4bb59b"
server_key_iv: "d328ef12cbca49c8afd061ac60b85b21"
server_key_version: "encryptionkey@0"
I20260809 23:13:19.899806 10569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "e3fe17f48c5f4077a625b1de6decdce0"
format_stamp: "Formatted at 2026-08-09 23:13:19 on dist-test-slave-j2vl"
server_key: "6c98a8e6b052a58220be747eef4bb59b"
server_key_iv: "d328ef12cbca49c8afd061ac60b85b21"
server_key_version: "encryptionkey@0"
I20260809 23:13:19.901253 10569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:19.902226 10607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:19.902500 10569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:19.902632 10569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "e3fe17f48c5f4077a625b1de6decdce0"
format_stamp: "Formatted at 2026-08-09 23:13:19 on dist-test-slave-j2vl"
server_key: "6c98a8e6b052a58220be747eef4bb59b"
server_key_iv: "d328ef12cbca49c8afd061ac60b85b21"
server_key_version: "encryptionkey@0"
I20260809 23:13:19.902695 10569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:19.914309 10569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:19.916518 10569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:19.916640 10569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:19.921216 10569 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43117
I20260809 23:13:19.921305 10690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43117 every 8 connection(s)
I20260809 23:13:19.921693 10569 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:19.922768 10691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:19.923036 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10569
I20260809 23:13:19.923171 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:19.923388 8759 external_mini_cluster.cc:1428] Setting key 46b282cc9a788fa80a945e54c5619fb1
I20260809 23:13:19.926560 10691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0: Bootstrap starting.
I20260809 23:13:19.927333 10691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:19.927654 10691 log.cc:824] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:19.928462 10691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0: No bootstrap required, opened a new log
Aug 09 23:13:19 dist-test-slave-j2vl krb5kdc[10542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:13:19.931260 10691 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3fe17f48c5f4077a625b1de6decdce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260809 23:13:19.931452 10691 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:19.931512 10691 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3fe17f48c5f4077a625b1de6decdce0, State: Initialized, Role: FOLLOWER
I20260809 23:13:19.931737 10691 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3fe17f48c5f4077a625b1de6decdce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260809 23:13:19.931839 10691 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:19.931887 10691 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:19.931941 10691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:19.932701 10691 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3fe17f48c5f4077a625b1de6decdce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260809 23:13:19.932825 10691 leader_election.cc:302] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3fe17f48c5f4077a625b1de6decdce0; no voters:
I20260809 23:13:19.933050 10691 leader_election.cc:288] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:19.933085 10701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:19.933215 10701 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [term 1 LEADER]: Becoming Leader. State: Replica: e3fe17f48c5f4077a625b1de6decdce0, State: Running, Role: LEADER
I20260809 23:13:19.933411 10691 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:19.933411 10701 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3fe17f48c5f4077a625b1de6decdce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } }
I20260809 23:13:19.933977 10702 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3fe17f48c5f4077a625b1de6decdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3fe17f48c5f4077a625b1de6decdce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } } }
I20260809 23:13:19.934077 10702 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:19.934310 10703 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3fe17f48c5f4077a625b1de6decdce0. Latest consensus state: current_term: 1 leader_uuid: "e3fe17f48c5f4077a625b1de6decdce0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3fe17f48c5f4077a625b1de6decdce0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43117 } } }
I20260809 23:13:19.934706 10703 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:19.934916 10712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:19.935786 10712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:19.936151 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:19.923997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43133 (local address 127.0.0.1:43117)
0809 23:13:19.924196 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:19.924208 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:19.924244 (+ 36us) server_negotiation.cc:407] Connection header received
0809 23:13:19.924752 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:19.924760 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:19.924830 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:19.924999 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:19.925870 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:19.927254 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:19.928138 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:19.928599 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:19.928691 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:19.931622 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:19.931627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:19.931627 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:19.931639 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:19.933979 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:19.934502 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:19.934506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:19.934506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:19.934649 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:19.934918 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:19.934920 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:19.934920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:19.935360 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:19.935534 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:19.935680 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 23:13:19.935977 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":90,"threads_started":1}
I20260809 23:13:19.937201 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:19.937551 10712 catalog_manager.cc:1347] Generated new cluster ID: 9d6e3ea447b04d4c87cf6024ce8ef521
I20260809 23:13:19.937608 10712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:19.948271 10698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:19.941736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53083 (local address 127.0.0.1:43117)
0809 23:13:19.941832 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:19.941834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:19.942069 (+ 235us) server_negotiation.cc:407] Connection header received
0809 23:13:19.942151 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:19.942152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:19.942163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:19.942232 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:19.942806 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:19.943311 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:19.944019 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:19.944249 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:19.944326 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:19.945603 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:19.945605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:19.945606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:19.945613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:19.947008 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:19.947375 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:19.947376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:19.947377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:19.947434 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:19.947642 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:19.947643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:19.947644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:19.947796 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:19.947923 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:19.948085 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 23:13:19.948173 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 23:13:19.948513 10712 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:19.949326 10712 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:19.970252 10712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3fe17f48c5f4077a625b1de6decdce0: Generated new TSK 0
I20260809 23:13:19.970522 10712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:19.978811 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10569
2026-08-09T23:13:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1635 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:20 dist-test-slave-j2vl krb5kdc[10737](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:20 dist-test-slave-j2vl krb5kdc[10737](info): set up 2 sockets
Aug 09 23:13:20 dist-test-slave-j2vl krb5kdc[10737](info): commencing operation
krb5kdc: starting...
W20260809 23:13:21.306653 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:21 dist-test-slave-j2vl krb5kdc[10737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:21.449997 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:21.551942 10761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:21.552186 10761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:21.552214 10761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:21.553545 10761 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:21.553604 10761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:21.553619 10761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:21.553632 10761 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:21.553645 10761 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:21.555536 10761 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:21.555826 10761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:21.556120 10761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:21.561754 10772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:21.561911 10770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:21.561800 10774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:21.561982 10761 server_base.cc:1034] running on GCE node
I20260809 23:13:21.562256 10761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:21.562592 10761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:21.563791 10761 hybrid_clock.cc:648] HybridClock initialized: now 1786317201563773 us; error 57 us; skew 500 ppm
Aug 09 23:13:21 dist-test-slave-j2vl krb5kdc[10737](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317201, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:21.567255 10761 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:21.567764 10761 webserver.cc:460] Webserver started at http://127.0.0.1:42749/ using document root <none> and password file <none>
I20260809 23:13:21.568009 10761 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:21.568074 10761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:21.568222 10761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:21.569403 10761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "c2e7e27d56264ed092f216bd583cbfa2"
format_stamp: "Formatted at 2026-08-09 23:13:21 on dist-test-slave-j2vl"
server_key: "5ba5a9541628fd175063e655cd65653c"
server_key_iv: "ac63c323cf2354ec715b5ea030f4e0f3"
server_key_version: "encryptionkey@0"
I20260809 23:13:21.569794 10761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "c2e7e27d56264ed092f216bd583cbfa2"
format_stamp: "Formatted at 2026-08-09 23:13:21 on dist-test-slave-j2vl"
server_key: "5ba5a9541628fd175063e655cd65653c"
server_key_iv: "ac63c323cf2354ec715b5ea030f4e0f3"
server_key_version: "encryptionkey@0"
I20260809 23:13:21.571238 10761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:21.572073 10785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:21.572254 10761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:21.572365 10761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "c2e7e27d56264ed092f216bd583cbfa2"
format_stamp: "Formatted at 2026-08-09 23:13:21 on dist-test-slave-j2vl"
server_key: "5ba5a9541628fd175063e655cd65653c"
server_key_iv: "ac63c323cf2354ec715b5ea030f4e0f3"
server_key_version: "encryptionkey@0"
I20260809 23:13:21.572439 10761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:21.586258 10761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:21.588711 10761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:21.588905 10761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:21.593655 10761 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33611
I20260809 23:13:21.593701 10871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33611 every 8 connection(s)
I20260809 23:13:21.594089 10761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:21.595145 10872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:21.596812 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10761
I20260809 23:13:21.596954 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:21.597173 8759 external_mini_cluster.cc:1428] Setting key 718f837e3c02d73d7a49cc7fe74f4f16
I20260809 23:13:21.598089 10872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2: Bootstrap starting.
I20260809 23:13:21.598747 10872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:21.599099 10872 log.cc:824] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:21.599871 10872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2: No bootstrap required, opened a new log
I20260809 23:13:21.602509 10872 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e7e27d56264ed092f216bd583cbfa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33611 } }
I20260809 23:13:21.602685 10872 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:21.602711 10872 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2e7e27d56264ed092f216bd583cbfa2, State: Initialized, Role: FOLLOWER
I20260809 23:13:21.602857 10872 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e7e27d56264ed092f216bd583cbfa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33611 } }
I20260809 23:13:21.602952 10872 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:21.603026 10872 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:21.603085 10872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:21.603350 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:21.597809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58627 (local address 127.0.0.1:33611)
0809 23:13:21.598007 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:21.598015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:21.598042 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:13:21.598550 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:21.598558 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:21.598607 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:21.598725 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:21.598981 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:21.600114 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:21.601233 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:21.601697 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:21.601781 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:21.602563 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:21.602566 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:21.602567 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:21.602799 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:21.602854 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:21.602870 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:21.602973 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:21.603055 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 23:13:21.603131 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260809 23:13:21.603828 10872 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e7e27d56264ed092f216bd583cbfa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33611 } }
I20260809 23:13:21.603957 10872 leader_election.cc:302] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2e7e27d56264ed092f216bd583cbfa2; no voters:
I20260809 23:13:21.604138 10872 leader_election.cc:288] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:21.604175 10884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:21.604341 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:21.604403 10884 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [term 1 LEADER]: Becoming Leader. State: Replica: c2e7e27d56264ed092f216bd583cbfa2, State: Running, Role: LEADER
I20260809 23:13:21.604416 10872 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:21.604732 10884 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e7e27d56264ed092f216bd583cbfa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33611 } }
I20260809 23:13:21.605257 10886 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2e7e27d56264ed092f216bd583cbfa2. Latest consensus state: current_term: 1 leader_uuid: "c2e7e27d56264ed092f216bd583cbfa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e7e27d56264ed092f216bd583cbfa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33611 } } }
I20260809 23:13:21.605257 10887 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2e7e27d56264ed092f216bd583cbfa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2e7e27d56264ed092f216bd583cbfa2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33611 } } }
I20260809 23:13:21.605410 10886 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:21.605432 10887 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:21.605865 10903 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:21.606411 10903 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:21.608036 10903 catalog_manager.cc:1347] Generated new cluster ID: 9327502dcc8549b387689ef4a6995a35
I20260809 23:13:21.608091 10903 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:21.611153 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:21.607362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40537 (local address 127.0.0.1:33611)
0809 23:13:21.607396 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:21.607398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:21.607516 (+ 118us) server_negotiation.cc:407] Connection header received
0809 23:13:21.607649 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:21.607650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:21.607664 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:21.607722 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:21.607900 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:21.608611 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:21.609697 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:21.609998 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:21.610050 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:21.610838 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:21.610839 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:21.610840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:21.610844 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:21.610858 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:21.610862 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:21.610907 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:21.611052 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 23:13:21.611074 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:13:21.625679 10903 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:21.626497 10903 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:21.631798 10903 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2e7e27d56264ed092f216bd583cbfa2: Generated new TSK 0
I20260809 23:13:21.632025 10903 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:21.637151 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10761
2026-08-09T23:13:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1663 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:21 dist-test-slave-j2vl krb5kdc[10924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:21 dist-test-slave-j2vl krb5kdc[10924](info): set up 2 sockets
Aug 09 23:13:21 dist-test-slave-j2vl krb5kdc[10924](info): commencing operation
krb5kdc: starting...
W20260809 23:13:22.959378 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:23 dist-test-slave-j2vl krb5kdc[10924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317203, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:23.100654 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:23.186794 10948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:23.187068 10948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:23.187110 10948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:23.188356 10948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:23.188436 10948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:23.188463 10948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:23.188494 10948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:23.188519 10948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:23.190220 10948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:23.190554 10948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:23.190806 10948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:23.196575 10956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:23.196679 10954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:23.196807 10961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:23.196874 10948 server_base.cc:1034] running on GCE node
I20260809 23:13:23.197188 10948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:23.197551 10948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:23.198729 10948 hybrid_clock.cc:648] HybridClock initialized: now 1786317203198689 us; error 50 us; skew 500 ppm
Aug 09 23:13:23 dist-test-slave-j2vl krb5kdc[10924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317203, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:23.201752 10948 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:23.202391 10948 webserver.cc:460] Webserver started at http://127.0.0.1:43341/ using document root <none> and password file <none>
I20260809 23:13:23.202625 10948 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:23.202672 10948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:23.202770 10948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:23.203826 10948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "fcfacaf4d5b74d91a2220301de9db085"
format_stamp: "Formatted at 2026-08-09 23:13:23 on dist-test-slave-j2vl"
server_key: "c40505bf7494af814a2564241bc5100b"
server_key_iv: "4f5d6dc17879a3de5d439e46b027f8db"
server_key_version: "encryptionkey@0"
I20260809 23:13:23.204161 10948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "fcfacaf4d5b74d91a2220301de9db085"
format_stamp: "Formatted at 2026-08-09 23:13:23 on dist-test-slave-j2vl"
server_key: "c40505bf7494af814a2564241bc5100b"
server_key_iv: "4f5d6dc17879a3de5d439e46b027f8db"
server_key_version: "encryptionkey@0"
I20260809 23:13:23.205602 10948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 23:13:23.206405 10969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:23.206597 10948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:23.206694 10948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "fcfacaf4d5b74d91a2220301de9db085"
format_stamp: "Formatted at 2026-08-09 23:13:23 on dist-test-slave-j2vl"
server_key: "c40505bf7494af814a2564241bc5100b"
server_key_iv: "4f5d6dc17879a3de5d439e46b027f8db"
server_key_version: "encryptionkey@0"
I20260809 23:13:23.206758 10948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:23.210309 10948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:23.212627 10948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:23.212756 10948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:23.217156 10948 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39071
I20260809 23:13:23.217231 11064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39071 every 8 connection(s)
I20260809 23:13:23.217567 10948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:23.218676 11065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:23.221472 11065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085: Bootstrap starting.
I20260809 23:13:23.222157 11065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:23.222460 11065 log.cc:824] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:23.223245 11065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085: No bootstrap required, opened a new log
I20260809 23:13:23.225075 11065 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfacaf4d5b74d91a2220301de9db085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260809 23:13:23.225240 11065 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:23.225267 11065 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcfacaf4d5b74d91a2220301de9db085, State: Initialized, Role: FOLLOWER
I20260809 23:13:23.225461 11065 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfacaf4d5b74d91a2220301de9db085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260809 23:13:23.225535 11065 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:23.225559 11065 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:23.225592 11065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:23.226186 11065 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfacaf4d5b74d91a2220301de9db085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260809 23:13:23.226279 11065 leader_election.cc:302] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcfacaf4d5b74d91a2220301de9db085; no voters:
I20260809 23:13:23.226444 11065 leader_election.cc:288] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:23.226584 11068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:23.226725 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 10948
I20260809 23:13:23.226802 11068 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [term 1 LEADER]: Becoming Leader. State: Replica: fcfacaf4d5b74d91a2220301de9db085, State: Running, Role: LEADER
I20260809 23:13:23.226872 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:23.226933 11068 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfacaf4d5b74d91a2220301de9db085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } }
I20260809 23:13:23.227123 8759 external_mini_cluster.cc:1428] Setting key ee2f2f955ebe85ab600f4e0e31ef3a21
I20260809 23:13:23.226805 11065 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:23.227424 11075 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcfacaf4d5b74d91a2220301de9db085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfacaf4d5b74d91a2220301de9db085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260809 23:13:23.227735 11075 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:23.227576 11076 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcfacaf4d5b74d91a2220301de9db085. Latest consensus state: current_term: 1 leader_uuid: "fcfacaf4d5b74d91a2220301de9db085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcfacaf4d5b74d91a2220301de9db085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39071 } } }
I20260809 23:13:23.228176 11087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:23.228243 11076 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:23.228895 11087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:23.230700 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:23.227827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33623 (local address 127.0.0.1:39071)
0809 23:13:23.228299 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:23.228311 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:23.228363 (+ 52us) server_negotiation.cc:407] Connection header received
0809 23:13:23.228514 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:23.228521 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:23.228560 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:23.228725 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:23.230003 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:23.230008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:23.230008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:23.230013 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:23.230056 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:23.230069 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:23.230171 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:23.230459 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 23:13:23.230507 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":77,"threads_started":1}
I20260809 23:13:23.231043 11087 catalog_manager.cc:1347] Generated new cluster ID: 2b299212c597409d953769d0ed7e27bc
I20260809 23:13:23.231132 11087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:23.231794 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:23.234973 11088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:23.233713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37019 (local address 127.0.0.1:39071)
0809 23:13:23.233755 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:23.233757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:23.233920 (+ 163us) server_negotiation.cc:407] Connection header received
0809 23:13:23.233978 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:23.233979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:23.233986 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:23.234037 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:23.234737 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:23.234738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:23.234739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:23.234743 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:23.234760 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:23.234765 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:23.234826 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:23.234887 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 23:13:23.234901 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:13:23.247067 11087 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:23.247299 11087 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:23.263782 11087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcfacaf4d5b74d91a2220301de9db085: Generated new TSK 0
I20260809 23:13:23.264067 11087 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:23.273902 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 10948
2026-08-09T23:13:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1623 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:23 dist-test-slave-j2vl krb5kdc[11124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:23 dist-test-slave-j2vl krb5kdc[11124](info): set up 2 sockets
Aug 09 23:13:23 dist-test-slave-j2vl krb5kdc[11124](info): commencing operation
krb5kdc: starting...
W20260809 23:13:24.604272 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:24 dist-test-slave-j2vl krb5kdc[11124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:24.752519 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46195
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:24.840016 11146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:24.840312 11146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:24.840360 11146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:24.841641 11146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:24.841722 11146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:24.841755 11146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:24.841787 11146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:24.841812 11146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:24.843433 11146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:24.843768 11146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:24.844022 11146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:24.849695 11155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:24.849699 11156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:24.849699 11146 server_base.cc:1034] running on GCE node
W20260809 23:13:24.849925 11160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:24.850369 11146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:24.850729 11146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:24.851939 11146 hybrid_clock.cc:648] HybridClock initialized: now 1786317204851919 us; error 44 us; skew 500 ppm
Aug 09 23:13:24 dist-test-slave-j2vl krb5kdc[11124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317204, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:24.854923 11146 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:24.855558 11146 webserver.cc:460] Webserver started at http://127.0.0.1:38379/ using document root <none> and password file <none>
I20260809 23:13:24.855844 11146 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:24.855895 11146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:24.856037 11146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:24.857201 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "902b2f43ee8040eab871e2bb66e019e2"
format_stamp: "Formatted at 2026-08-09 23:13:24 on dist-test-slave-j2vl"
server_key: "b9adc4224b680406f49d65963dd75ebe"
server_key_iv: "da2542117357e55b383ea9343d47f5c6"
server_key_version: "encryptionkey@0"
I20260809 23:13:24.857707 11146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "902b2f43ee8040eab871e2bb66e019e2"
format_stamp: "Formatted at 2026-08-09 23:13:24 on dist-test-slave-j2vl"
server_key: "b9adc4224b680406f49d65963dd75ebe"
server_key_iv: "da2542117357e55b383ea9343d47f5c6"
server_key_version: "encryptionkey@0"
I20260809 23:13:24.859302 11146 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:24.860392 11177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:24.860656 11146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:24.860785 11146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "902b2f43ee8040eab871e2bb66e019e2"
format_stamp: "Formatted at 2026-08-09 23:13:24 on dist-test-slave-j2vl"
server_key: "b9adc4224b680406f49d65963dd75ebe"
server_key_iv: "da2542117357e55b383ea9343d47f5c6"
server_key_version: "encryptionkey@0"
I20260809 23:13:24.860915 11146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:24.876559 11146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:24.879029 11146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:24.879163 11146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:24.883986 11146 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46195
I20260809 23:13:24.884032 11266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46195 every 8 connection(s)
I20260809 23:13:24.884459 11146 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:24.885648 11267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:24.888330 11267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2: Bootstrap starting.
I20260809 23:13:24.888995 11267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:24.889434 11267 log.cc:824] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:24.889669 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11146
I20260809 23:13:24.889823 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:24.890084 8759 external_mini_cluster.cc:1428] Setting key 9387ee0861422e2cdeb74fbc17fd7494
I20260809 23:13:24.890501 11267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2: No bootstrap required, opened a new log
I20260809 23:13:24.892809 11267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902b2f43ee8040eab871e2bb66e019e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46195 } }
I20260809 23:13:24.893020 11267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:24.893110 11267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 902b2f43ee8040eab871e2bb66e019e2, State: Initialized, Role: FOLLOWER
I20260809 23:13:24.893299 11267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902b2f43ee8040eab871e2bb66e019e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46195 } }
I20260809 23:13:24.893440 11267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:24.893491 11267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:24.893548 11267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:24.894650 11267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902b2f43ee8040eab871e2bb66e019e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46195 } }
I20260809 23:13:24.894796 11267 leader_election.cc:302] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 902b2f43ee8040eab871e2bb66e019e2; no voters:
I20260809 23:13:24.895036 11267 leader_election.cc:288] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:24.895066 11275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:24.895183 11275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [term 1 LEADER]: Becoming Leader. State: Replica: 902b2f43ee8040eab871e2bb66e019e2, State: Running, Role: LEADER
I20260809 23:13:24.895351 11267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:24.895331 11275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902b2f43ee8040eab871e2bb66e019e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46195 } }
I20260809 23:13:24.895824 11279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 902b2f43ee8040eab871e2bb66e019e2. Latest consensus state: current_term: 1 leader_uuid: "902b2f43ee8040eab871e2bb66e019e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902b2f43ee8040eab871e2bb66e019e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46195 } } }
I20260809 23:13:24.895938 11279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:24.896293 11278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "902b2f43ee8040eab871e2bb66e019e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "902b2f43ee8040eab871e2bb66e019e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46195 } } }
I20260809 23:13:24.896369 11278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:24.896711 11286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:24.897629 11286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:13:24 dist-test-slave-j2vl krb5kdc[11124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:13:24.899780 11286 catalog_manager.cc:1347] Generated new cluster ID: 84ed249c9bdc4e48bbda32287ffcc9d5
I20260809 23:13:24.899835 11286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:24.904208 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:24.890842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:56483 (local address 127.0.0.1:46195)
0809 23:13:24.891078 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:24.891088 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:24.891123 (+ 35us) server_negotiation.cc:407] Connection header received
0809 23:13:24.892796 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:24.892806 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:24.892876 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:24.893080 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:24.894080 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:24.895427 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:24.896553 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:24.897101 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:24.897231 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:24.900287 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:24.900292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:24.900292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:24.900307 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:24.902234 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:24.902853 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:24.902857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:24.902857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:24.903000 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:24.903257 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:24.903258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:24.903258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:24.903548 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:24.903749 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:24.903839 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 23:13:24.903953 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":120,"threads_started":1}
I20260809 23:13:24.905668 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:24.914316 11273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:24.907297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:56027 (local address 127.0.0.1:46195)
0809 23:13:24.907340 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:24.907342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:24.907568 (+ 226us) server_negotiation.cc:407] Connection header received
0809 23:13:24.907646 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:24.907647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:24.907658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:24.907715 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:24.908289 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:24.908806 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:24.909663 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:24.909887 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:24.909981 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:24.911296 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:24.911298 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:24.911298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:24.911306 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:24.912955 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:24.913430 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:24.913432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:24.913433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:24.913500 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:24.913743 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:24.913745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:24.913745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:24.913872 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:24.914016 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:24.914132 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 23:13:24.914207 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 23:13:24.933830 11286 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:24.934602 11286 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:24.944345 11286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 902b2f43ee8040eab871e2bb66e019e2: Generated new TSK 0
I20260809 23:13:24.944602 11286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:24.953701 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11146
W20260809 23:13:24.967617 11121 connection.cc:535] client connection to 127.0.0.1:46195 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46195): Connection reset by peer (error 104)
2026-08-09T23:13:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1682 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:25 dist-test-slave-j2vl krb5kdc[11326](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:25 dist-test-slave-j2vl krb5kdc[11326](info): set up 2 sockets
Aug 09 23:13:25 dist-test-slave-j2vl krb5kdc[11326](info): commencing operation
krb5kdc: starting...
W20260809 23:13:26.279129 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.297s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:26 dist-test-slave-j2vl krb5kdc[11326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:26.430817 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:26.527741 11344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:26.528093 11344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:26.528141 11344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:26.529690 11344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:26.529793 11344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:26.529821 11344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:26.529857 11344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:26.529889 11344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:26.531750 11344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:26.532111 11344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:26.532389 11344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:26.538972 11354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:26.538972 11360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:26.539000 11352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:26.539855 11344 server_base.cc:1034] running on GCE node
I20260809 23:13:26.540150 11344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:26.540500 11344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:26.541692 11344 hybrid_clock.cc:648] HybridClock initialized: now 1786317206541679 us; error 35 us; skew 500 ppm
Aug 09 23:13:26 dist-test-slave-j2vl krb5kdc[11326](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317206, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:26.548714 11344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:26.549314 11344 webserver.cc:460] Webserver started at http://127.0.0.1:36573/ using document root <none> and password file <none>
I20260809 23:13:26.549640 11344 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:26.549721 11344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:26.549901 11344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:26.551182 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "d1878bb09ac34cf0977f3043a16251f3"
format_stamp: "Formatted at 2026-08-09 23:13:26 on dist-test-slave-j2vl"
server_key: "34822b95adaa808b24a357c30d5950c1"
server_key_iv: "d29520b7f47d678293f725619b7c5f6e"
server_key_version: "encryptionkey@0"
I20260809 23:13:26.551616 11344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "d1878bb09ac34cf0977f3043a16251f3"
format_stamp: "Formatted at 2026-08-09 23:13:26 on dist-test-slave-j2vl"
server_key: "34822b95adaa808b24a357c30d5950c1"
server_key_iv: "d29520b7f47d678293f725619b7c5f6e"
server_key_version: "encryptionkey@0"
I20260809 23:13:26.553570 11344 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 23:13:26.554701 11371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:26.555195 11344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:26.555326 11344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "d1878bb09ac34cf0977f3043a16251f3"
format_stamp: "Formatted at 2026-08-09 23:13:26 on dist-test-slave-j2vl"
server_key: "34822b95adaa808b24a357c30d5950c1"
server_key_iv: "d29520b7f47d678293f725619b7c5f6e"
server_key_version: "encryptionkey@0"
I20260809 23:13:26.555426 11344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:26.576084 11344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:26.578699 11344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:26.578893 11344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:26.583549 11344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41309
I20260809 23:13:26.583612 11477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41309 every 8 connection(s)
I20260809 23:13:26.584033 11344 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:26.585289 11479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:26.587811 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11344
I20260809 23:13:26.587972 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:26.588135 11479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3: Bootstrap starting.
I20260809 23:13:26.588172 8759 external_mini_cluster.cc:1428] Setting key 1ea801bf8780aaa10e897de927737aeb
I20260809 23:13:26.588967 11479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:26.589401 11479 log.cc:824] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:26.590444 11479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3: No bootstrap required, opened a new log
I20260809 23:13:26.592728 11479 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1878bb09ac34cf0977f3043a16251f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260809 23:13:26.592994 11479 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:26.593052 11479 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1878bb09ac34cf0977f3043a16251f3, State: Initialized, Role: FOLLOWER
I20260809 23:13:26.593197 11479 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1878bb09ac34cf0977f3043a16251f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260809 23:13:26.593287 11479 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:26.593333 11479 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:26.593415 11479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:26.594322 11479 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1878bb09ac34cf0977f3043a16251f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260809 23:13:26.594422 11479 leader_election.cc:302] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1878bb09ac34cf0977f3043a16251f3; no voters:
I20260809 23:13:26.594630 11479 leader_election.cc:288] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:26.594681 11487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:26.594897 11487 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [term 1 LEADER]: Becoming Leader. State: Replica: d1878bb09ac34cf0977f3043a16251f3, State: Running, Role: LEADER
I20260809 23:13:26.595034 11487 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1878bb09ac34cf0977f3043a16251f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } }
I20260809 23:13:26.595042 11479 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:26.595355 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:26.588879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:57917 (local address 127.0.0.1:41309)
0809 23:13:26.589113 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:26.589126 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:26.589160 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:13:26.589840 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:26.589851 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:26.589941 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:26.590095 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:26.590434 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:26.591585 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:26.592776 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:26.593343 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:26.593493 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:26.594258 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:26.594262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:26.594262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:26.594508 (+ 246us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:26.594559 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:26.594575 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:26.594682 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:26.594787 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 23:13:26.594926 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":115,"threads_started":1}
I20260809 23:13:26.595593 11495 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1878bb09ac34cf0977f3043a16251f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1878bb09ac34cf0977f3043a16251f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } } }
I20260809 23:13:26.595788 11495 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:26.595717 11496 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1878bb09ac34cf0977f3043a16251f3. Latest consensus state: current_term: 1 leader_uuid: "d1878bb09ac34cf0977f3043a16251f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1878bb09ac34cf0977f3043a16251f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41309 } } }
I20260809 23:13:26.596249 11496 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:26.596426 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:26.596491 11498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:26.597704 11498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:26.599607 11498 catalog_manager.cc:1347] Generated new cluster ID: 00d6980ddc1344378c6e21550aec38bb
I20260809 23:13:26.599669 11498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:26.605367 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:26.598131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:53339 (local address 127.0.0.1:41309)
0809 23:13:26.598176 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:26.598178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:26.601701 (+ 3523us) server_negotiation.cc:407] Connection header received
0809 23:13:26.601770 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:26.601771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:26.601784 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:26.601861 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:26.602024 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:26.602804 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:26.603854 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:26.604195 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:26.604276 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:26.605073 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:26.605075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:26.605075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:26.605081 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:26.605099 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:26.605104 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:26.605171 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:26.605242 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 23:13:26.605254 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:13:26.631502 11498 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:26.632409 11498 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:26.639075 11498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1878bb09ac34cf0977f3043a16251f3: Generated new TSK 0
I20260809 23:13:26.639346 11498 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:26.648248 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11344
W20260809 23:13:26.663422 11321 connection.cc:535] client connection to 127.0.0.1:41309 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41309): Connection reset by peer (error 104)
2026-08-09T23:13:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1695 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:26 dist-test-slave-j2vl krb5kdc[11533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:26 dist-test-slave-j2vl krb5kdc[11533](info): set up 2 sockets
Aug 09 23:13:26 dist-test-slave-j2vl krb5kdc[11533](info): commencing operation
krb5kdc: starting...
W20260809 23:13:27.976504 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:28 dist-test-slave-j2vl krb5kdc[11533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:28.117321 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:28.203745 11556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:28.204042 11556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:28.204093 11556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:28.205466 11556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:28.205555 11556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:28.205580 11556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:28.205605 11556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:28.205636 11556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:28.207306 11556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:28.207652 11556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:28.207934 11556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:28.213725 11562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:28.213670 11566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:28.213714 11569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:28.213955 11556 server_base.cc:1034] running on GCE node
I20260809 23:13:28.214277 11556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:28.214576 11556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:28.215749 11556 hybrid_clock.cc:648] HybridClock initialized: now 1786317208215714 us; error 52 us; skew 500 ppm
Aug 09 23:13:28 dist-test-slave-j2vl krb5kdc[11533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317208, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:28.218582 11556 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:28.219084 11556 webserver.cc:460] Webserver started at http://127.0.0.1:43885/ using document root <none> and password file <none>
I20260809 23:13:28.219341 11556 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:28.219391 11556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:28.219532 11556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:28.220657 11556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "8c2a3c95d101496bb65e0df3cf88057c"
format_stamp: "Formatted at 2026-08-09 23:13:28 on dist-test-slave-j2vl"
server_key: "ed7ed7ed30d99225a2dfd9d2045ca607"
server_key_iv: "dc567a355643a4025fb1e76f15f80b04"
server_key_version: "encryptionkey@0"
I20260809 23:13:28.221052 11556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "8c2a3c95d101496bb65e0df3cf88057c"
format_stamp: "Formatted at 2026-08-09 23:13:28 on dist-test-slave-j2vl"
server_key: "ed7ed7ed30d99225a2dfd9d2045ca607"
server_key_iv: "dc567a355643a4025fb1e76f15f80b04"
server_key_version: "encryptionkey@0"
I20260809 23:13:28.222640 11556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:28.223608 11582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:28.223839 11556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:28.223939 11556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "8c2a3c95d101496bb65e0df3cf88057c"
format_stamp: "Formatted at 2026-08-09 23:13:28 on dist-test-slave-j2vl"
server_key: "ed7ed7ed30d99225a2dfd9d2045ca607"
server_key_iv: "dc567a355643a4025fb1e76f15f80b04"
server_key_version: "encryptionkey@0"
I20260809 23:13:28.224004 11556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:28.227449 11556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:28.229972 11556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:28.230098 11556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:28.234601 11556 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41219
I20260809 23:13:28.234688 11671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41219 every 8 connection(s)
I20260809 23:13:28.235062 11556 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:28.236228 11672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:28.238804 11672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c: Bootstrap starting.
I20260809 23:13:28.239481 11672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:28.239800 11672 log.cc:824] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:28.240623 11672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c: No bootstrap required, opened a new log
I20260809 23:13:28.242516 11672 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2a3c95d101496bb65e0df3cf88057c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41219 } }
I20260809 23:13:28.242693 11672 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:28.242743 11672 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c2a3c95d101496bb65e0df3cf88057c, State: Initialized, Role: FOLLOWER
I20260809 23:13:28.242952 11672 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2a3c95d101496bb65e0df3cf88057c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41219 } }
I20260809 23:13:28.243064 11672 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:28.243114 11672 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:28.243167 11672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:28.243772 11672 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2a3c95d101496bb65e0df3cf88057c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41219 } }
I20260809 23:13:28.243899 11672 leader_election.cc:302] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c2a3c95d101496bb65e0df3cf88057c; no voters:
I20260809 23:13:28.243939 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11556
I20260809 23:13:28.244045 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:28.244134 11672 leader_election.cc:288] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:28.244221 11676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:28.244266 8759 external_mini_cluster.cc:1428] Setting key c754fdc71af3b80f88f5f3f82e768c2d
I20260809 23:13:28.244438 11676 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [term 1 LEADER]: Becoming Leader. State: Replica: 8c2a3c95d101496bb65e0df3cf88057c, State: Running, Role: LEADER
I20260809 23:13:28.244444 11672 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:28.244616 11676 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2a3c95d101496bb65e0df3cf88057c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41219 } }
I20260809 23:13:28.245241 11677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c2a3c95d101496bb65e0df3cf88057c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2a3c95d101496bb65e0df3cf88057c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41219 } } }
I20260809 23:13:28.245400 11677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:28.245730 11679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c2a3c95d101496bb65e0df3cf88057c. Latest consensus state: current_term: 1 leader_uuid: "8c2a3c95d101496bb65e0df3cf88057c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c2a3c95d101496bb65e0df3cf88057c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41219 } } }
I20260809 23:13:28.245813 11679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:28.246393 11684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:28.246954 11684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:28.247476 11682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:28.245169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:59629 (local address 127.0.0.1:41219)
0809 23:13:28.245411 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:28.245422 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:28.245459 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:13:28.245592 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:28.245600 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:28.245639 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:28.245795 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:28.246960 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:28.246964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:28.246965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:28.246977 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:28.247041 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:28.247058 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:28.247128 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:28.247203 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 23:13:28.247256 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":81,"threads_started":1}
I20260809 23:13:28.248589 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:28.248672 11684 catalog_manager.cc:1347] Generated new cluster ID: eeb6dc9f55fe4df2979ca3e0441499cf
I20260809 23:13:28.248715 11684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:28.251341 11682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:28.250128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:37489 (local address 127.0.0.1:41219)
0809 23:13:28.250173 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:28.250176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:28.250321 (+ 145us) server_negotiation.cc:407] Connection header received
0809 23:13:28.250400 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:28.250401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:28.250407 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:28.250458 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:28.251108 (+ 650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:28.251109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:28.251110 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:28.251114 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:28.251129 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:28.251134 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:28.251184 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:28.251237 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 23:13:28.251253 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 23:13:28.260418 11684 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:28.260670 11684 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:28.266927 11684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c2a3c95d101496bb65e0df3cf88057c: Generated new TSK 0
I20260809 23:13:28.267159 11684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:28.273226 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11556
2026-08-09T23:13:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1627 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:28 dist-test-slave-j2vl krb5kdc[11720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:28 dist-test-slave-j2vl krb5kdc[11720](info): set up 2 sockets
Aug 09 23:13:28 dist-test-slave-j2vl krb5kdc[11720](info): commencing operation
krb5kdc: starting...
W20260809 23:13:29.589151 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:29 dist-test-slave-j2vl krb5kdc[11720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:29.733438 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:29.821305 11742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:29.821601 11742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:29.821645 11742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:29.822948 11742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:29.823029 11742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:29.823056 11742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:29.823087 11742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:29.823120 11742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:29.824725 11742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:29.825038 11742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:29.825295 11742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:29.830924 11749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:29.830932 11756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:29.831071 11751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:29.831069 11742 server_base.cc:1034] running on GCE node
I20260809 23:13:29.831614 11742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:29.831923 11742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:29.833132 11742 hybrid_clock.cc:648] HybridClock initialized: now 1786317209833120 us; error 33 us; skew 500 ppm
Aug 09 23:13:29 dist-test-slave-j2vl krb5kdc[11720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:29.836392 11742 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:29.836889 11742 webserver.cc:460] Webserver started at http://127.0.0.1:46031/ using document root <none> and password file <none>
I20260809 23:13:29.837162 11742 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:29.837234 11742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:29.837441 11742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:29.838639 11742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "d36255ee19124ad5ae130d3a28cecfef"
format_stamp: "Formatted at 2026-08-09 23:13:29 on dist-test-slave-j2vl"
server_key: "1052cba1ba3b0fe5c67702712f8af8a7"
server_key_iv: "565ca6c625df6de475a17fca7e14265b"
server_key_version: "encryptionkey@0"
I20260809 23:13:29.839044 11742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "d36255ee19124ad5ae130d3a28cecfef"
format_stamp: "Formatted at 2026-08-09 23:13:29 on dist-test-slave-j2vl"
server_key: "1052cba1ba3b0fe5c67702712f8af8a7"
server_key_iv: "565ca6c625df6de475a17fca7e14265b"
server_key_version: "encryptionkey@0"
I20260809 23:13:29.840646 11742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:29.841586 11771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:29.841789 11742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:29.841907 11742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "d36255ee19124ad5ae130d3a28cecfef"
format_stamp: "Formatted at 2026-08-09 23:13:29 on dist-test-slave-j2vl"
server_key: "1052cba1ba3b0fe5c67702712f8af8a7"
server_key_iv: "565ca6c625df6de475a17fca7e14265b"
server_key_version: "encryptionkey@0"
I20260809 23:13:29.841993 11742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:29.864171 11742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:29.866822 11742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:29.867035 11742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:29.872251 11742 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34251
I20260809 23:13:29.872313 11862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34251 every 8 connection(s)
I20260809 23:13:29.872745 11742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:29.874171 11866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:29.876989 11866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef: Bootstrap starting.
I20260809 23:13:29.877633 11866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:29.877897 11866 log.cc:824] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:29.879099 11866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef: No bootstrap required, opened a new log
I20260809 23:13:29.880524 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11742
I20260809 23:13:29.880756 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:29.881006 8759 external_mini_cluster.cc:1428] Setting key 3a78e18b901125cfec5d285b05a0d28d
I20260809 23:13:29.881927 11866 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36255ee19124ad5ae130d3a28cecfef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260809 23:13:29.882136 11866 raft_consensus.cc:374] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:29.882174 11866 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36255ee19124ad5ae130d3a28cecfef, State: Initialized, Role: FOLLOWER
I20260809 23:13:29.882336 11866 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36255ee19124ad5ae130d3a28cecfef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260809 23:13:29.882436 11866 raft_consensus.cc:388] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:29.882470 11866 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:29.882519 11866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:29.883882 11866 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36255ee19124ad5ae130d3a28cecfef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260809 23:13:29.884021 11866 leader_election.cc:302] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d36255ee19124ad5ae130d3a28cecfef; no voters:
I20260809 23:13:29.884204 11866 leader_election.cc:288] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:29.884289 11874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:29.884477 11866 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:29.884542 11874 raft_consensus.cc:686] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [term 1 LEADER]: Becoming Leader. State: Replica: d36255ee19124ad5ae130d3a28cecfef, State: Running, Role: LEADER
I20260809 23:13:29.884687 11874 consensus_queue.cc:237] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36255ee19124ad5ae130d3a28cecfef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } }
I20260809 23:13:29.885205 11881 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d36255ee19124ad5ae130d3a28cecfef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36255ee19124ad5ae130d3a28cecfef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } } }
I20260809 23:13:29.885303 11881 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:29.885528 11885 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36255ee19124ad5ae130d3a28cecfef. Latest consensus state: current_term: 1 leader_uuid: "d36255ee19124ad5ae130d3a28cecfef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d36255ee19124ad5ae130d3a28cecfef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34251 } } }
I20260809 23:13:29.885593 11885 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:29.885933 11895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:29.886813 11895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 23:13:29 dist-test-slave-j2vl krb5kdc[11720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 23:13:29.889006 11895 catalog_manager.cc:1347] Generated new cluster ID: 99458bd84e2f4ea1ad43d03564efb012
I20260809 23:13:29.889061 11895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:29.895409 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:29.881746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:54987 (local address 127.0.0.1:34251)
0809 23:13:29.881936 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:29.881947 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:29.881986 (+ 39us) server_negotiation.cc:407] Connection header received
0809 23:13:29.882108 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:29.882116 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:29.882182 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:29.882334 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:29.883131 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:29.884573 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:29.885857 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:29.886491 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:29.886604 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:29.889456 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:29.889460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:29.889460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:29.889474 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:29.892307 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:29.893302 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:29.893315 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:29.893315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:29.893654 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:29.894021 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:29.894022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:29.894023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:29.894434 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:29.894633 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:29.894815 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 23:13:29.895151 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260809 23:13:29.897087 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:29.904384 11895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:29.905922 11895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:29.906872 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:29.898862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:58973 (local address 127.0.0.1:34251)
0809 23:13:29.898903 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:29.898904 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:29.898915 (+ 11us) server_negotiation.cc:407] Connection header received
0809 23:13:29.898976 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:29.898976 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:29.898987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:29.899045 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:29.899710 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:29.900492 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:29.901529 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:29.901886 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:29.901966 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:29.903505 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:29.903507 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:29.903507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:29.903516 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:29.905502 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:29.905922 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:29.905923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:29.905923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:29.906059 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:29.906311 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:29.906312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:29.906312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:29.906436 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:29.906570 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:29.906691 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 23:13:29.906756 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:13:29.918550 11895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d36255ee19124ad5ae130d3a28cecfef: Generated new TSK 0
I20260809 23:13:29.918824 11895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:29.926425 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11742
W20260809 23:13:29.952987 11717 connection.cc:535] client connection to 127.0.0.1:34251 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34251): Connection reset by peer (error 104)
2026-08-09T23:13:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1662 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:30 dist-test-slave-j2vl krb5kdc[11928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:30 dist-test-slave-j2vl krb5kdc[11928](info): set up 2 sockets
Aug 09 23:13:30 dist-test-slave-j2vl krb5kdc[11928](info): commencing operation
krb5kdc: starting...
W20260809 23:13:31.268558 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.301s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:31 dist-test-slave-j2vl krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:31.409039 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:31.495687 11953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:31.496028 11953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:31.496079 11953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:31.497509 11953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:31.497594 11953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:31.497619 11953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:31.497640 11953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:31.497674 11953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:31.499428 11953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39533
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:31.499764 11953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:31.500038 11953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:31.505544 11964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:31.505599 11962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:31.505571 11969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:31.505677 11953 server_base.cc:1034] running on GCE node
I20260809 23:13:31.506132 11953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:31.506493 11953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:31.507699 11953 hybrid_clock.cc:648] HybridClock initialized: now 1786317211507674 us; error 41 us; skew 500 ppm
Aug 09 23:13:31 dist-test-slave-j2vl krb5kdc[11928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317211, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:31.510848 11953 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:31.511408 11953 webserver.cc:460] Webserver started at http://127.0.0.1:45493/ using document root <none> and password file <none>
I20260809 23:13:31.511690 11953 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:31.511766 11953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:31.511924 11953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:31.513103 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "28abe78c3a0b414c87199a3e29b1d1ee"
format_stamp: "Formatted at 2026-08-09 23:13:31 on dist-test-slave-j2vl"
server_key: "c597c16a3e8f7d3f0e0a829299af72c6"
server_key_iv: "810e985038e28bef5864ead08413a7fa"
server_key_version: "encryptionkey@0"
I20260809 23:13:31.513540 11953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "28abe78c3a0b414c87199a3e29b1d1ee"
format_stamp: "Formatted at 2026-08-09 23:13:31 on dist-test-slave-j2vl"
server_key: "c597c16a3e8f7d3f0e0a829299af72c6"
server_key_iv: "810e985038e28bef5864ead08413a7fa"
server_key_version: "encryptionkey@0"
I20260809 23:13:31.515190 11953 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:31.516291 11985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:31.516650 11953 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:31.516784 11953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "28abe78c3a0b414c87199a3e29b1d1ee"
format_stamp: "Formatted at 2026-08-09 23:13:31 on dist-test-slave-j2vl"
server_key: "c597c16a3e8f7d3f0e0a829299af72c6"
server_key_iv: "810e985038e28bef5864ead08413a7fa"
server_key_version: "encryptionkey@0"
I20260809 23:13:31.516868 11953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:31.538174 11953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:31.540604 11953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:31.540807 11953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:31.545581 11953 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39533
I20260809 23:13:31.545622 12076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39533 every 8 connection(s)
I20260809 23:13:31.546038 11953 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:31.547070 12078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:31.549579 12078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee: Bootstrap starting.
I20260809 23:13:31.550206 12078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:31.550467 12078 log.cc:824] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:31.551232 12078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee: No bootstrap required, opened a new log
I20260809 23:13:31.552910 12078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260809 23:13:31.553071 12078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:31.553095 12078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28abe78c3a0b414c87199a3e29b1d1ee, State: Initialized, Role: FOLLOWER
I20260809 23:13:31.553251 12078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260809 23:13:31.553359 12078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:31.553433 12078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:31.553471 12078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:31.554076 12078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260809 23:13:31.554171 12078 leader_election.cc:302] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28abe78c3a0b414c87199a3e29b1d1ee; no voters:
I20260809 23:13:31.554333 12078 leader_election.cc:288] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:31.554410 12082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:31.554618 12082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [term 1 LEADER]: Becoming Leader. State: Replica: 28abe78c3a0b414c87199a3e29b1d1ee, State: Running, Role: LEADER
I20260809 23:13:31.554651 12078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:31.554726 12082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260809 23:13:31.555133 12087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28abe78c3a0b414c87199a3e29b1d1ee. Latest consensus state: current_term: 1 leader_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } } }
I20260809 23:13:31.555277 12083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28abe78c3a0b414c87199a3e29b1d1ee" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } } }
I20260809 23:13:31.555285 12087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:31.555399 12083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:31.555764 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 11953
I20260809 23:13:31.555917 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:31.556259 8759 external_mini_cluster.cc:1428] Setting key efbdeb4014a557152420a8b8b38558ec
W20260809 23:13:31.556990 12111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:31.557052 12111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:31.557118 12102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:31.557299 12102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:31.559065 12102 catalog_manager.cc:1347] Generated new cluster ID: 27e912f6a46c4aec8aa20c6720bfea59
I20260809 23:13:31.559125 12102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:31.566802 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:31.556841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:38177 (local address 127.0.0.1:39533)
0809 23:13:31.557184 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:31.557195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:31.557229 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:13:31.557404 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:31.557413 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:31.557477 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:31.557631 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:31.557875 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:31.559095 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:31.560203 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:31.560819 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:31.560907 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:31.566008 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:31.566013 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:31.566014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:31.566258 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:31.566303 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:31.566320 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:31.566427 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:31.566513 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 23:13:31.566546 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":74,"threads_started":1}
I20260809 23:13:31.568092 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:31.573335 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:31.569634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:48715 (local address 127.0.0.1:39533)
0809 23:13:31.569679 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:31.569681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:31.569771 (+ 90us) server_negotiation.cc:407] Connection header received
0809 23:13:31.569820 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:31.569821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:31.569832 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:31.569886 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:31.570025 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:31.570789 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:31.571839 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:31.572164 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:31.572232 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:31.573010 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:31.573011 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:31.573011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:31.573017 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:31.573035 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:31.573040 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:31.573107 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:31.573231 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 23:13:31.573247 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:13:31.573768 12102 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:31.574397 12102 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:31.591523 12102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28abe78c3a0b414c87199a3e29b1d1ee: Generated new TSK 0
I20260809 23:13:31.591809 12102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:31.598382 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 11953
W20260809 23:13:31.615870 11926 connection.cc:535] client connection to 127.0.0.1:39533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39533): Connection reset by peer (error 104)
2026-08-09T23:13:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1662 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:31 dist-test-slave-j2vl krb5kdc[12126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:31 dist-test-slave-j2vl krb5kdc[12126](info): set up 2 sockets
Aug 09 23:13:31 dist-test-slave-j2vl krb5kdc[12126](info): commencing operation
krb5kdc: starting...
W20260809 23:13:32.906790 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:32 dist-test-slave-j2vl krb5kdc[12126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 23:13:33.040678 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:33.130304 12152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:33.130584 12152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:33.130628 12152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:33.131814 12152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:33.131886 12152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:33.131911 12152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:33.131943 12152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:33.131968 12152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:33.133610 12152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:33.133908 12152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:33.134167 12152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:33.139679 12169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:33.139688 12164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:33.139679 12163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:33.140127 12152 server_base.cc:1034] running on GCE node
I20260809 23:13:33.140445 12152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:33.140759 12152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:33.141937 12152 hybrid_clock.cc:648] HybridClock initialized: now 1786317213141913 us; error 40 us; skew 500 ppm
Aug 09 23:13:33 dist-test-slave-j2vl krb5kdc[12126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317213, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:33.144690 12152 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 23:13:33.145222 12152 webserver.cc:460] Webserver started at http://127.0.0.1:34515/ using document root <none> and password file <none>
I20260809 23:13:33.145539 12152 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:33.145613 12152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:33.145766 12152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:33.146994 12152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "47045129b771435fa5db4422815037db"
format_stamp: "Formatted at 2026-08-09 23:13:33 on dist-test-slave-j2vl"
server_key: "7c5cbf9bc845483fc6b922c66c5f6951"
server_key_iv: "748498b9322f643629488959d8d23778"
server_key_version: "encryptionkey@0"
I20260809 23:13:33.147378 12152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "47045129b771435fa5db4422815037db"
format_stamp: "Formatted at 2026-08-09 23:13:33 on dist-test-slave-j2vl"
server_key: "7c5cbf9bc845483fc6b922c66c5f6951"
server_key_iv: "748498b9322f643629488959d8d23778"
server_key_version: "encryptionkey@0"
I20260809 23:13:33.148916 12152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:33.149839 12185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:33.150125 12152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:13:33.150247 12152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "47045129b771435fa5db4422815037db"
format_stamp: "Formatted at 2026-08-09 23:13:33 on dist-test-slave-j2vl"
server_key: "7c5cbf9bc845483fc6b922c66c5f6951"
server_key_iv: "748498b9322f643629488959d8d23778"
server_key_version: "encryptionkey@0"
I20260809 23:13:33.150327 12152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:33.153893 12152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:33.156038 12152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:33.156200 12152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:33.160923 12152 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34239
I20260809 23:13:33.160954 12275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34239 every 8 connection(s)
I20260809 23:13:33.161423 12152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:33.162513 12277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:33.164924 12277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db: Bootstrap starting.
I20260809 23:13:33.165606 12277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:33.165913 12277 log.cc:824] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:33.166642 12277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db: No bootstrap required, opened a new log
I20260809 23:13:33.166988 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12152
I20260809 23:13:33.167110 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:33.167296 8759 external_mini_cluster.cc:1428] Setting key 567695b1e26f6215ec9308ec4675437b
I20260809 23:13:33.169076 12277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47045129b771435fa5db4422815037db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260809 23:13:33.169261 12277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:33.169301 12277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47045129b771435fa5db4422815037db, State: Initialized, Role: FOLLOWER
I20260809 23:13:33.169533 12277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47045129b771435fa5db4422815037db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260809 23:13:33.169947 12277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:33.169996 12277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:33.170060 12277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:33.170580 12284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:33.167912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:55323 (local address 127.0.0.1:34239)
0809 23:13:33.168120 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:33.168130 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:33.168166 (+ 36us) server_negotiation.cc:407] Connection header received
0809 23:13:33.169022 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:33.169033 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:33.169070 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:33.169230 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:33.170145 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:33.170148 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:33.170149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:33.170154 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:33.170204 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:33.170215 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:33.170291 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:33.170361 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 23:13:33.170408 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260809 23:13:33.170843 12277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47045129b771435fa5db4422815037db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260809 23:13:33.170953 12277 leader_election.cc:302] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47045129b771435fa5db4422815037db; no voters:
I20260809 23:13:33.171118 12277 leader_election.cc:288] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:33.171176 12287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:33.171329 12287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [term 1 LEADER]: Becoming Leader. State: Replica: 47045129b771435fa5db4422815037db, State: Running, Role: LEADER
I20260809 23:13:33.171402 12277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:33.171475 12287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47045129b771435fa5db4422815037db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } }
I20260809 23:13:33.171525 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:33.172793 12290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47045129b771435fa5db4422815037db. Latest consensus state: current_term: 1 leader_uuid: "47045129b771435fa5db4422815037db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47045129b771435fa5db4422815037db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } } }
I20260809 23:13:33.172894 12290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:33.173164 12289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47045129b771435fa5db4422815037db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47045129b771435fa5db4422815037db" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34239 } } }
I20260809 23:13:33.173261 12289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db [sys.catalog]: This master's current role is: LEADER
W20260809 23:13:33.173457 12318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:33.173566 12318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:33.173709 12319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:33.173894 12319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:33.174875 12284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:33.173583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:38781 (local address 127.0.0.1:34239)
0809 23:13:33.173628 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:33.173630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:33.173761 (+ 131us) server_negotiation.cc:407] Connection header received
0809 23:13:33.173911 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:33.173912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:33.173917 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:33.173969 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:33.174631 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:33.174632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:33.174632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:33.174637 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:33.174652 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:33.174656 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:33.174738 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:33.174798 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 23:13:33.174811 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 23:13:33.175645 12319 catalog_manager.cc:1347] Generated new cluster ID: ec364b5aa26f44b896a97e8f645b166c
I20260809 23:13:33.175704 12319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:33.191591 12319 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:33.191815 12319 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:33.197022 12319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47045129b771435fa5db4422815037db: Generated new TSK 0
I20260809 23:13:33.197263 12319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:33.202603 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12152
2026-08-09T23:13:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:33 dist-test-slave-j2vl krb5kdc[12328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:33 dist-test-slave-j2vl krb5kdc[12328](info): set up 2 sockets
Aug 09 23:13:33 dist-test-slave-j2vl krb5kdc[12328](info): commencing operation
krb5kdc: starting...
W20260809 23:13:34.559955 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:34 dist-test-slave-j2vl krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:34.705428 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42879
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:39979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:42879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:34.793021 12353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:34.793254 12353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:34.793279 12353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:34.794535 12353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:34.794586 12353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:34.794601 12353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:34.794615 12353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:34.794626 12353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:34.796201 12353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:39979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:42879
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:42879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:34.796430 12353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:34.796619 12353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:34.801844 12362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:34.801873 12365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:34.801851 12363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:34.802181 12353 server_base.cc:1034] running on GCE node
I20260809 23:13:34.802467 12353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:34.802784 12353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:34.803966 12353 hybrid_clock.cc:648] HybridClock initialized: now 1786317214803945 us; error 44 us; skew 500 ppm
Aug 09 23:13:34 dist-test-slave-j2vl krb5kdc[12328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317214, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:34.806821 12353 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:34.807332 12353 webserver.cc:460] Webserver started at http://127.8.141.254:35355/ using document root <none> and password file <none>
I20260809 23:13:34.807579 12353 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:34.807626 12353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:34.807770 12353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:34.808879 12353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "5771d88becce4dd985a9bfc20c25b876"
format_stamp: "Formatted at 2026-08-09 23:13:34 on dist-test-slave-j2vl"
server_key: "ba1a71348c32cc6b297bec3b8dd2135f"
server_key_iv: "eb3fc0ebaddeb841578252b985a30b73"
server_key_version: "encryptionkey@0"
I20260809 23:13:34.809248 12353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "5771d88becce4dd985a9bfc20c25b876"
format_stamp: "Formatted at 2026-08-09 23:13:34 on dist-test-slave-j2vl"
server_key: "ba1a71348c32cc6b297bec3b8dd2135f"
server_key_iv: "eb3fc0ebaddeb841578252b985a30b73"
server_key_version: "encryptionkey@0"
I20260809 23:13:34.810736 12353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:34.811630 12374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:34.811923 12353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:34.812037 12353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "5771d88becce4dd985a9bfc20c25b876"
format_stamp: "Formatted at 2026-08-09 23:13:34 on dist-test-slave-j2vl"
server_key: "ba1a71348c32cc6b297bec3b8dd2135f"
server_key_iv: "eb3fc0ebaddeb841578252b985a30b73"
server_key_version: "encryptionkey@0"
I20260809 23:13:34.812096 12353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:34.821492 12353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:34.824537 12353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:34.824673 12353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:34.829113 12353 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:42879
I20260809 23:13:34.829203 12468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:42879 every 8 connection(s)
I20260809 23:13:34.829620 12353 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:34.830658 12470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:34.832057 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12353
I20260809 23:13:34.832231 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:34.832444 8759 external_mini_cluster.cc:1428] Setting key 90305b1ea618e6410351c611a7f83975
I20260809 23:13:34.833691 12470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876: Bootstrap starting.
I20260809 23:13:34.834411 12470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:34.834805 12470 log.cc:824] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:34.835680 12470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876: No bootstrap required, opened a new log
I20260809 23:13:34.838101 12470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5771d88becce4dd985a9bfc20c25b876" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42879 } }
I20260809 23:13:34.838310 12470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:34.838359 12470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5771d88becce4dd985a9bfc20c25b876, State: Initialized, Role: FOLLOWER
I20260809 23:13:34.838521 12470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5771d88becce4dd985a9bfc20c25b876" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42879 } }
I20260809 23:13:34.838613 12470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:34.838647 12470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:34.838697 12470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:34.839352 12470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5771d88becce4dd985a9bfc20c25b876" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42879 } }
I20260809 23:13:34.839452 12470 leader_election.cc:302] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5771d88becce4dd985a9bfc20c25b876; no voters:
I20260809 23:13:34.839685 12470 leader_election.cc:288] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:34.839710 12478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:34.839879 12478 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [term 1 LEADER]: Becoming Leader. State: Replica: 5771d88becce4dd985a9bfc20c25b876, State: Running, Role: LEADER
I20260809 23:13:34.839962 12478 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5771d88becce4dd985a9bfc20c25b876" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42879 } }
I20260809 23:13:34.840025 12470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:34.840313 12480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5771d88becce4dd985a9bfc20c25b876" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5771d88becce4dd985a9bfc20c25b876" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42879 } } }
I20260809 23:13:34.840392 12480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [sys.catalog]: This master's current role is: LEADER
Aug 09 23:13:34 dist-test-slave-j2vl krb5kdc[12328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:34.840302 12478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5771d88becce4dd985a9bfc20c25b876. Latest consensus state: current_term: 1 leader_uuid: "5771d88becce4dd985a9bfc20c25b876" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5771d88becce4dd985a9bfc20c25b876" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 42879 } } }
I20260809 23:13:34.840598 12478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:34.840728 12485 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:34.841476 12485 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:34.843199 12485 catalog_manager.cc:1347] Generated new cluster ID: 03770a32459d4116b1ff6a278babdeaa
I20260809 23:13:34.843259 12485 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:34.846315 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:34.833124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53847 (local address 127.8.141.254:42879)
0809 23:13:34.833395 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:34.833407 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:34.833597 (+ 190us) server_negotiation.cc:407] Connection header received
0809 23:13:34.834123 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:34.834130 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:34.834183 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:34.834311 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:34.835258 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:34.836327 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:34.837481 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:34.837936 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:34.838031 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:34.841264 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:34.841268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:34.841269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:34.841290 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:34.843730 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:34.844615 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:34.844619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:34.844619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:34.844749 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:34.845031 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:34.845033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:34.845033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:34.845438 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:34.845619 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:34.845756 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 23:13:34.846037 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
I20260809 23:13:34.847462 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:34.857616 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:34.849133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39089 (local address 127.8.141.254:42879)
0809 23:13:34.849183 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:34.849185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:34.849218 (+ 33us) server_negotiation.cc:407] Connection header received
0809 23:13:34.849277 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:34.849278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:34.849290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:34.849420 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:34.850076 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:34.850905 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:34.851945 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:34.852316 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:34.852398 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:34.854314 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:34.854315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:34.854316 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:34.854326 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:34.856134 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:34.856641 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:34.856643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:34.856643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:34.856716 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:34.856968 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:34.856969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:34.856969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:34.857123 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:34.857237 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:34.857434 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 23:13:34.857510 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:13:34.860627 12485 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:34.861419 12485 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:34.871793 12485 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5771d88becce4dd985a9bfc20c25b876: Generated new TSK 0
I20260809 23:13:34.872051 12485 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:34.876912 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12353
W20260809 23:13:34.888365 12324 connection.cc:535] client connection to 127.8.141.254:42879 recv error: Network error: failed to read from TLS socket (remote: 127.8.141.254:42879): Connection reset by peer (error 104)
2026-08-09T23:13:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1675 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:34 dist-test-slave-j2vl krb5kdc[12523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:34 dist-test-slave-j2vl krb5kdc[12523](info): set up 2 sockets
Aug 09 23:13:34 dist-test-slave-j2vl krb5kdc[12523](info): commencing operation
krb5kdc: starting...
W20260809 23:13:36.207463 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:36 dist-test-slave-j2vl krb5kdc[12523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:36.362322 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44219
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:37399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:44219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:36.455088 12546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:36.455562 12546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:36.455596 12546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:36.457546 12546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:36.457628 12546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:36.457646 12546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:36.457661 12546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:36.457675 12546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:36.459846 12546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:37399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:44219
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:44219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:36.460166 12546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:36.460470 12546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:36.468384 12552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:36.468387 12556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:36.468750 12553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:36.468854 12546 server_base.cc:1034] running on GCE node
I20260809 23:13:36.469290 12546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:36.469729 12546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:36.470963 12546 hybrid_clock.cc:648] HybridClock initialized: now 1786317216470938 us; error 38 us; skew 500 ppm
Aug 09 23:13:36 dist-test-slave-j2vl krb5kdc[12523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317216, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:36.474515 12546 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:36.475116 12546 webserver.cc:460] Webserver started at http://127.8.141.254:44339/ using document root <none> and password file <none>
I20260809 23:13:36.475358 12546 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:36.475409 12546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:36.475525 12546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:36.476755 12546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "8ddddc3faa174d92b1e401301a1f380b"
format_stamp: "Formatted at 2026-08-09 23:13:36 on dist-test-slave-j2vl"
server_key: "08168b7783cbf64272bfa947509f303b"
server_key_iv: "7ed79fe686824068e698e5eb667908af"
server_key_version: "encryptionkey@0"
I20260809 23:13:36.477106 12546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "8ddddc3faa174d92b1e401301a1f380b"
format_stamp: "Formatted at 2026-08-09 23:13:36 on dist-test-slave-j2vl"
server_key: "08168b7783cbf64272bfa947509f303b"
server_key_iv: "7ed79fe686824068e698e5eb667908af"
server_key_version: "encryptionkey@0"
I20260809 23:13:36.478780 12546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:36.479945 12569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:36.480342 12546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:36.480466 12546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "8ddddc3faa174d92b1e401301a1f380b"
format_stamp: "Formatted at 2026-08-09 23:13:36 on dist-test-slave-j2vl"
server_key: "08168b7783cbf64272bfa947509f303b"
server_key_iv: "7ed79fe686824068e698e5eb667908af"
server_key_version: "encryptionkey@0"
I20260809 23:13:36.480584 12546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:36.489913 12546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:36.492659 12546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:36.492859 12546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:36.498045 12546 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:44219
I20260809 23:13:36.498180 12651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:44219 every 8 connection(s)
I20260809 23:13:36.498623 12546 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:36.499583 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12546
I20260809 23:13:36.499728 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:36.499838 12652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:36.499961 8759 external_mini_cluster.cc:1428] Setting key 223ca15da9e1dc685895836d7ab51a11
I20260809 23:13:36.505268 12652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b: Bootstrap starting.
I20260809 23:13:36.506048 12652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:36.506381 12652 log.cc:824] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:36.507337 12652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b: No bootstrap required, opened a new log
I20260809 23:13:36.508617 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:36.500829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59009 (local address 127.8.141.254:44219)
0809 23:13:36.501202 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:36.501219 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:36.501397 (+ 178us) server_negotiation.cc:407] Connection header received
0809 23:13:36.503126 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:36.503137 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:36.503212 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:36.503404 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:36.503669 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:36.505061 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:36.506049 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:36.506686 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:36.506783 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:36.507736 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:36.507741 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:36.507742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:36.508040 (+ 298us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:36.508082 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:36.508100 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:36.508201 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:36.508350 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 23:13:36.508381 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":164,"threads_started":1}
I20260809 23:13:36.510226 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:36.510322 12652 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddddc3faa174d92b1e401301a1f380b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44219 } }
I20260809 23:13:36.510538 12652 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:36.510588 12652 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddddc3faa174d92b1e401301a1f380b, State: Initialized, Role: FOLLOWER
I20260809 23:13:36.510735 12652 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddddc3faa174d92b1e401301a1f380b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44219 } }
I20260809 23:13:36.510810 12652 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:36.510835 12652 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:36.510907 12652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:36.511673 12652 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddddc3faa174d92b1e401301a1f380b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44219 } }
I20260809 23:13:36.511777 12652 leader_election.cc:302] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ddddc3faa174d92b1e401301a1f380b; no voters:
I20260809 23:13:36.512039 12652 leader_election.cc:288] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:36.512117 12664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:36.512355 12652 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:36.512382 12664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [term 1 LEADER]: Becoming Leader. State: Replica: 8ddddc3faa174d92b1e401301a1f380b, State: Running, Role: LEADER
I20260809 23:13:36.512562 12664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddddc3faa174d92b1e401301a1f380b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44219 } }
I20260809 23:13:36.513029 12672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ddddc3faa174d92b1e401301a1f380b. Latest consensus state: current_term: 1 leader_uuid: "8ddddc3faa174d92b1e401301a1f380b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddddc3faa174d92b1e401301a1f380b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44219 } } }
I20260809 23:13:36.513197 12672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:36.513497 12671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ddddc3faa174d92b1e401301a1f380b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ddddc3faa174d92b1e401301a1f380b" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 44219 } } }
I20260809 23:13:36.513898 12671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b [sys.catalog]: This master's current role is: LEADER
W20260809 23:13:36.515082 12694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:36.515154 12694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:36.515264 12674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:36.515574 12674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:36.516640 12655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:36.512275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54107 (local address 127.8.141.254:44219)
0809 23:13:36.512328 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:36.512331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:36.512363 (+ 32us) server_negotiation.cc:407] Connection header received
0809 23:13:36.512388 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:36.512389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:36.512400 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:36.512457 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:36.512690 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:36.513615 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:36.514915 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:36.515240 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:36.515317 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:36.516333 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:36.516335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:36.516335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:36.516341 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:36.516357 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:36.516362 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:36.516456 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:36.516529 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 23:13:36.516540 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 23:13:36.517650 12674 catalog_manager.cc:1347] Generated new cluster ID: 9227818031c8466f81689b3d58440d65
I20260809 23:13:36.517714 12674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:36.526293 12674 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:36.526961 12674 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:36.530603 12674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ddddc3faa174d92b1e401301a1f380b: Generated new TSK 0
I20260809 23:13:36.530822 12674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:36.534101 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12546
W20260809 23:13:36.552004 12521 connection.cc:535] client connection to 127.8.141.254:44219 recv error: Network error: failed to read from TLS socket (remote: 127.8.141.254:44219): Connection reset by peer (error 104)
2026-08-09T23:13:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1663 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:36 dist-test-slave-j2vl krb5kdc[12704](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:36 dist-test-slave-j2vl krb5kdc[12704](info): set up 2 sockets
Aug 09 23:13:36 dist-test-slave-j2vl krb5kdc[12704](info): commencing operation
krb5kdc: starting...
W20260809 23:13:38.028595 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.462s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:38 dist-test-slave-j2vl krb5kdc[12704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:38.199383 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:37221
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:37221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:38.316526 12724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:38.316895 12724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:38.316983 12724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:38.318625 12724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:38.318742 12724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:38.318782 12724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:38.318812 12724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:38.318843 12724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:38.321069 12724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:37221
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:38.321487 12724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:38.321803 12724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:38.329846 12732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:38.329846 12735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:38.329847 12733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:38.330965 12724 server_base.cc:1034] running on GCE node
I20260809 23:13:38.331403 12724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:38.331903 12724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:38.333253 12724 hybrid_clock.cc:648] HybridClock initialized: now 1786317218333179 us; error 47 us; skew 500 ppm
Aug 09 23:13:38 dist-test-slave-j2vl krb5kdc[12704](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317218, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:38.337471 12724 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:38.338076 12724 webserver.cc:460] Webserver started at http://127.8.141.254:35783/ using document root <none> and password file <none>
I20260809 23:13:38.338367 12724 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:38.338505 12724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:38.338820 12724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:38.340158 12724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "f394fcebc2b745a6856e9ca3e3f8626d"
format_stamp: "Formatted at 2026-08-09 23:13:38 on dist-test-slave-j2vl"
server_key: "fe1600d81460bb5d8e36ee9adff69a69"
server_key_iv: "a8c7e63cd0d3477a1ed658820da912ea"
server_key_version: "encryptionkey@0"
I20260809 23:13:38.340811 12724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "f394fcebc2b745a6856e9ca3e3f8626d"
format_stamp: "Formatted at 2026-08-09 23:13:38 on dist-test-slave-j2vl"
server_key: "fe1600d81460bb5d8e36ee9adff69a69"
server_key_iv: "a8c7e63cd0d3477a1ed658820da912ea"
server_key_version: "encryptionkey@0"
I20260809 23:13:38.342361 12724 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 23:13:38.343400 12743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:38.343688 12724 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:38.343822 12724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "f394fcebc2b745a6856e9ca3e3f8626d"
format_stamp: "Formatted at 2026-08-09 23:13:38 on dist-test-slave-j2vl"
server_key: "fe1600d81460bb5d8e36ee9adff69a69"
server_key_iv: "a8c7e63cd0d3477a1ed658820da912ea"
server_key_version: "encryptionkey@0"
I20260809 23:13:38.343962 12724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:38.348331 12724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:38.350992 12724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:38.351157 12724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:38.356014 12724 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:37221
I20260809 23:13:38.356088 12824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:37221 every 8 connection(s)
I20260809 23:13:38.356519 12724 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:38.357803 12825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:38.357946 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12724
I20260809 23:13:38.358124 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:38.358398 8759 external_mini_cluster.cc:1428] Setting key d43c2af23e4a9177a41cc4b0f5dcb043
I20260809 23:13:38.362444 12825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d: Bootstrap starting.
I20260809 23:13:38.363924 12825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:38.364450 12825 log.cc:824] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:38.364992 12830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:38.359145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56965 (local address 127.8.141.254:37221)
0809 23:13:38.361002 (+ 1857us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:38.361013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:38.361050 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:13:38.361944 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:38.361986 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:38.362100 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:38.362360 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:38.364134 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:38.364139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:38.364139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:38.364145 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:38.364257 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:38.364279 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:38.364363 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:38.364528 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 23:13:38.364583 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1794,"thread_start_us":69,"threads_started":1}
I20260809 23:13:38.366551 12825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d: No bootstrap required, opened a new log
I20260809 23:13:38.367089 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:38.369117 12825 raft_consensus.cc:348] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37221 } }
I20260809 23:13:38.369472 12825 raft_consensus.cc:374] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:38.369601 12825 raft_consensus.cc:729] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f394fcebc2b745a6856e9ca3e3f8626d, State: Initialized, Role: FOLLOWER
I20260809 23:13:38.369774 12825 consensus_queue.cc:260] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37221 } }
I20260809 23:13:38.369901 12825 raft_consensus.cc:388] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:38.369990 12825 raft_consensus.cc:482] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:38.370155 12825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:38.371114 12825 raft_consensus.cc:504] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37221 } }
I20260809 23:13:38.371277 12825 leader_election.cc:302] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f394fcebc2b745a6856e9ca3e3f8626d; no voters:
I20260809 23:13:38.371497 12825 leader_election.cc:288] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:38.371611 12842 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:38.371803 12825 sys_catalog.cc:564] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:38.371827 12842 raft_consensus.cc:686] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [term 1 LEADER]: Becoming Leader. State: Replica: f394fcebc2b745a6856e9ca3e3f8626d, State: Running, Role: LEADER
I20260809 23:13:38.372152 12842 consensus_queue.cc:237] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37221 } }
I20260809 23:13:38.372751 12844 sys_catalog.cc:455] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f394fcebc2b745a6856e9ca3e3f8626d. Latest consensus state: current_term: 1 leader_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37221 } } }
I20260809 23:13:38.372864 12844 sys_catalog.cc:458] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:38.372953 12843 sys_catalog.cc:455] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f394fcebc2b745a6856e9ca3e3f8626d" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 37221 } } }
I20260809 23:13:38.373035 12843 sys_catalog.cc:458] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:38.373099 12860 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:38.374006 12860 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:38.375801 12830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:38.369453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51019 (local address 127.8.141.254:37221)
0809 23:13:38.369544 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:38.369551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:38.373634 (+ 4083us) server_negotiation.cc:407] Connection header received
0809 23:13:38.374000 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:38.374001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:38.374008 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:38.374042 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:38.375548 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:38.375550 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:38.375551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:38.375556 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:38.375572 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:38.375577 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:38.375608 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:38.375717 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 23:13:38.375733 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:13:38.377872 12860 catalog_manager.cc:1347] Generated new cluster ID: 567a84c040ad4f89a03831b23a4f0d50
I20260809 23:13:38.378109 12860 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:38.399099 12860 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:38.399350 12860 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:38.408414 12860 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f394fcebc2b745a6856e9ca3e3f8626d: Generated new TSK 0
I20260809 23:13:38.408731 12860 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:38.417474 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12724
2026-08-09T23:13:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1883 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:38 dist-test-slave-j2vl krb5kdc[12882](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:38 dist-test-slave-j2vl krb5kdc[12882](info): set up 2 sockets
Aug 09 23:13:38 dist-test-slave-j2vl krb5kdc[12882](info): commencing operation
krb5kdc: starting...
W20260809 23:13:39.912793 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.462s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:40 dist-test-slave-j2vl krb5kdc[12882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:40.066689 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33605
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:41117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:33605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:40.164459 12902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:40.164693 12902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:40.164719 12902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:40.166030 12902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:40.166085 12902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:40.166100 12902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:40.166112 12902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:40.166124 12902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:40.167716 12902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:41117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:33605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:33605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:40.167994 12902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:40.168243 12902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:40.173826 12917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:40.173918 12925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:40.173938 12921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:40.173998 12902 server_base.cc:1034] running on GCE node
I20260809 23:13:40.174630 12902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:40.174964 12902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:40.176134 12902 hybrid_clock.cc:648] HybridClock initialized: now 1786317220176108 us; error 42 us; skew 500 ppm
Aug 09 23:13:40 dist-test-slave-j2vl krb5kdc[12882](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317220, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:40.178969 12902 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:40.179487 12902 webserver.cc:460] Webserver started at http://127.8.141.254:45735/ using document root <none> and password file <none>
I20260809 23:13:40.179724 12902 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:40.179792 12902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:40.179957 12902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:40.181072 12902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "5253208548ca45fcbba27afd25e529ec"
format_stamp: "Formatted at 2026-08-09 23:13:40 on dist-test-slave-j2vl"
server_key: "ace9e8943341b25c1b9ced1f6d2d64bb"
server_key_iv: "abc60401b84f4604bb570b5b0ed139a9"
server_key_version: "encryptionkey@0"
I20260809 23:13:40.181530 12902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "5253208548ca45fcbba27afd25e529ec"
format_stamp: "Formatted at 2026-08-09 23:13:40 on dist-test-slave-j2vl"
server_key: "ace9e8943341b25c1b9ced1f6d2d64bb"
server_key_iv: "abc60401b84f4604bb570b5b0ed139a9"
server_key_version: "encryptionkey@0"
I20260809 23:13:40.183081 12902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:40.184073 12936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:40.184340 12902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:13:40.184456 12902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "5253208548ca45fcbba27afd25e529ec"
format_stamp: "Formatted at 2026-08-09 23:13:40 on dist-test-slave-j2vl"
server_key: "ace9e8943341b25c1b9ced1f6d2d64bb"
server_key_iv: "abc60401b84f4604bb570b5b0ed139a9"
server_key_version: "encryptionkey@0"
I20260809 23:13:40.184551 12902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:40.192301 12902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:40.194841 12902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:40.195024 12902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:40.199520 12902 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:33605
I20260809 23:13:40.199563 13022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:33605 every 8 connection(s)
I20260809 23:13:40.199978 12902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:40.200907 13025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:40.203473 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 12902
I20260809 23:13:40.203567 13025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec: Bootstrap starting.
I20260809 23:13:40.203588 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:40.203804 8759 external_mini_cluster.cc:1428] Setting key 86c3c2be196b987631b6c73547074e91
I20260809 23:13:40.204393 13025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:40.204851 13025 log.cc:824] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:40.205724 13025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec: No bootstrap required, opened a new log
I20260809 23:13:40.208335 13025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5253208548ca45fcbba27afd25e529ec" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33605 } }
I20260809 23:13:40.208515 13025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:40.208540 13025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5253208548ca45fcbba27afd25e529ec, State: Initialized, Role: FOLLOWER
I20260809 23:13:40.208696 13025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5253208548ca45fcbba27afd25e529ec" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33605 } }
I20260809 23:13:40.208770 13025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:40.208823 13025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:40.208880 13025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:40.209731 13025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5253208548ca45fcbba27afd25e529ec" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33605 } }
I20260809 23:13:40.209852 13025 leader_election.cc:302] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5253208548ca45fcbba27afd25e529ec; no voters:
I20260809 23:13:40.210042 13025 leader_election.cc:288] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:40.210068 13033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:40.210196 13033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [term 1 LEADER]: Becoming Leader. State: Replica: 5253208548ca45fcbba27afd25e529ec, State: Running, Role: LEADER
I20260809 23:13:40.210284 13033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5253208548ca45fcbba27afd25e529ec" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33605 } }
I20260809 23:13:40.210335 13025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:40.210625 13033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5253208548ca45fcbba27afd25e529ec. Latest consensus state: current_term: 1 leader_uuid: "5253208548ca45fcbba27afd25e529ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5253208548ca45fcbba27afd25e529ec" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33605 } } }
I20260809 23:13:40.210747 13035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5253208548ca45fcbba27afd25e529ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5253208548ca45fcbba27afd25e529ec" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 33605 } } }
I20260809 23:13:40.210901 13035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:40.210762 13033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec [sys.catalog]: This master's current role is: LEADER
W20260809 23:13:40.212206 13069 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:40.212327 13069 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 23:13:40 dist-test-slave-j2vl krb5kdc[12882](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:40.212749 13046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:40.212949 13046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:40.215564 13046 catalog_manager.cc:1347] Generated new cluster ID: 447f2435794049809b8245fe902faa93
I20260809 23:13:40.215624 13046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:40.217216 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:40.204450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42019 (local address 127.8.141.254:33605)
0809 23:13:40.204685 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:40.204697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:40.204734 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:13:40.205395 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:40.205404 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:40.205484 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:40.205649 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:40.206474 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:40.207932 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:40.209058 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:40.209751 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:40.209875 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:40.213207 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:40.213212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:40.213212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:40.213225 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:40.215412 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:40.215918 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:40.215922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:40.215922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:40.216029 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:40.216254 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:40.216256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:40.216256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:40.216514 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:40.216683 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:40.216808 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 23:13:40.216980 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":101,"threads_started":1}
I20260809 23:13:40.218290 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:40.226959 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:40.219829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47773 (local address 127.8.141.254:33605)
0809 23:13:40.219869 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:40.219870 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:40.220071 (+ 201us) server_negotiation.cc:407] Connection header received
0809 23:13:40.220158 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:40.220159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:40.220171 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:40.220228 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:40.220800 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:40.221292 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:40.222030 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:40.222260 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:40.222335 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:40.223879 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:40.223881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:40.223882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:40.223890 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:40.225581 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:40.225990 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:40.225991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:40.225992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:40.226058 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:40.226333 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:40.226334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:40.226334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:40.226466 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:40.226579 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:40.226726 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 23:13:40.226826 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:13:40.229497 13046 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:40.230228 13046 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:40.239318 13046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5253208548ca45fcbba27afd25e529ec: Generated new TSK 0
I20260809 23:13:40.239569 13046 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:40.245712 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 12902
2026-08-09T23:13:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1831 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:40 dist-test-slave-j2vl krb5kdc[13088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:40 dist-test-slave-j2vl krb5kdc[13088](info): set up 2 sockets
Aug 09 23:13:40 dist-test-slave-j2vl krb5kdc[13088](info): commencing operation
krb5kdc: starting...
W20260809 23:13:41.579160 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:41 dist-test-slave-j2vl krb5kdc[13088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:41.730808 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:46323
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:46323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:41.828058 13111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:41.828382 13111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:41.828435 13111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:41.830195 13111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:41.830297 13111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:41.830325 13111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:41.830348 13111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:41.830379 13111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:41.832293 13111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:46323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:46323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:41.832654 13111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:41.832938 13111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:41.839475 13123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:41.839977 13119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:41.840764 13118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:41.841256 13111 server_base.cc:1034] running on GCE node
I20260809 23:13:41.841665 13111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:41.842043 13111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:41.843253 13111 hybrid_clock.cc:648] HybridClock initialized: now 1786317221843205 us; error 77 us; skew 500 ppm
Aug 09 23:13:41 dist-test-slave-j2vl krb5kdc[13088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317221, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:41.850652 13111 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:41.851359 13111 webserver.cc:460] Webserver started at http://127.8.141.254:42649/ using document root <none> and password file <none>
I20260809 23:13:41.851676 13111 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:41.851763 13111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:41.851971 13111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:41.853264 13111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "342f47e70d764e9bba30b1f183cf8965"
format_stamp: "Formatted at 2026-08-09 23:13:41 on dist-test-slave-j2vl"
server_key: "b7af5f9dc947f8aa14a22dc3240ac279"
server_key_iv: "b2e67ff72c59f836c408841e19fc0bcc"
server_key_version: "encryptionkey@0"
I20260809 23:13:41.853806 13111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "342f47e70d764e9bba30b1f183cf8965"
format_stamp: "Formatted at 2026-08-09 23:13:41 on dist-test-slave-j2vl"
server_key: "b7af5f9dc947f8aa14a22dc3240ac279"
server_key_iv: "b2e67ff72c59f836c408841e19fc0bcc"
server_key_version: "encryptionkey@0"
I20260809 23:13:41.855774 13111 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 23:13:41.857098 13133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:41.857571 13111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:41.857663 13111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "342f47e70d764e9bba30b1f183cf8965"
format_stamp: "Formatted at 2026-08-09 23:13:41 on dist-test-slave-j2vl"
server_key: "b7af5f9dc947f8aa14a22dc3240ac279"
server_key_iv: "b2e67ff72c59f836c408841e19fc0bcc"
server_key_version: "encryptionkey@0"
I20260809 23:13:41.857728 13111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:41.881299 13111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:41.884523 13111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:41.884665 13111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:41.889704 13111 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:46323
I20260809 23:13:41.889741 13223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:46323 every 8 connection(s)
I20260809 23:13:41.890166 13111 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:41.891175 13224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:41.893982 13224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965: Bootstrap starting.
I20260809 23:13:41.894731 13224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:41.895061 13224 log.cc:824] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:41.896872 13224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965: No bootstrap required, opened a new log
I20260809 23:13:41.899171 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13111
I20260809 23:13:41.899302 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:41.899523 8759 external_mini_cluster.cc:1428] Setting key 9d8575b7e36dd2803e8807e90e20e853
I20260809 23:13:41.899772 13224 raft_consensus.cc:348] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f47e70d764e9bba30b1f183cf8965" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46323 } }
I20260809 23:13:41.899971 13224 raft_consensus.cc:374] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:41.900039 13224 raft_consensus.cc:729] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342f47e70d764e9bba30b1f183cf8965, State: Initialized, Role: FOLLOWER
I20260809 23:13:41.900214 13224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f47e70d764e9bba30b1f183cf8965" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46323 } }
I20260809 23:13:41.900389 13224 raft_consensus.cc:388] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:41.900434 13224 raft_consensus.cc:482] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:41.900467 13224 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:41.901768 13224 raft_consensus.cc:504] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f47e70d764e9bba30b1f183cf8965" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46323 } }
I20260809 23:13:41.901881 13224 leader_election.cc:302] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 342f47e70d764e9bba30b1f183cf8965; no voters:
I20260809 23:13:41.902098 13224 leader_election.cc:288] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:41.902161 13231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:41.902401 13231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [term 1 LEADER]: Becoming Leader. State: Replica: 342f47e70d764e9bba30b1f183cf8965, State: Running, Role: LEADER
I20260809 23:13:41.902406 13224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:41.902598 13231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f47e70d764e9bba30b1f183cf8965" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46323 } }
I20260809 23:13:41.903049 13232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "342f47e70d764e9bba30b1f183cf8965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f47e70d764e9bba30b1f183cf8965" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46323 } } }
I20260809 23:13:41.903132 13232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:41.903594 13234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 342f47e70d764e9bba30b1f183cf8965. Latest consensus state: current_term: 1 leader_uuid: "342f47e70d764e9bba30b1f183cf8965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342f47e70d764e9bba30b1f183cf8965" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 46323 } } }
I20260809 23:13:41.903731 13234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:41.903976 13245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:41.904789 13245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:41.906558 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:41.900167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34277 (local address 127.8.141.254:46323)
0809 23:13:41.900418 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:41.900446 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:41.900491 (+ 45us) server_negotiation.cc:407] Connection header received
0809 23:13:41.900651 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:41.900658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:41.900705 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:41.900850 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:41.901319 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:41.902784 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:41.904092 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:41.904744 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:41.904856 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:41.905738 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:41.905743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:41.905743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:41.905974 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:41.906018 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:41.906034 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:41.906111 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:41.906227 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 23:13:41.906304 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":119,"threads_started":1}
I20260809 23:13:41.907003 13245 catalog_manager.cc:1347] Generated new cluster ID: f899f9f82e534929b64336f6660b0427
I20260809 23:13:41.907078 13245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:41.907685 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:41.918017 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:41.909644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35173 (local address 127.8.141.254:46323)
0809 23:13:41.909695 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:41.909698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:41.913835 (+ 4137us) server_negotiation.cc:407] Connection header received
0809 23:13:41.913881 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:41.913883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:41.913899 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:41.913971 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:41.914215 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:41.915398 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:41.916504 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:41.916826 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:41.916915 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:41.917730 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:41.917732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:41.917732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:41.917738 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:41.917756 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:41.917761 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:41.917824 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:41.917882 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 23:13:41.917915 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:13:41.944343 13245 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:41.945184 13245 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:41.952827 13245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 342f47e70d764e9bba30b1f183cf8965: Generated new TSK 0
I20260809 23:13:41.953264 13245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:41.960158 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13111
2026-08-09T23:13:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1704 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:42 dist-test-slave-j2vl krb5kdc[13276](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:42 dist-test-slave-j2vl krb5kdc[13276](info): set up 2 sockets
Aug 09 23:13:42 dist-test-slave-j2vl krb5kdc[13276](info): commencing operation
krb5kdc: starting...
W20260809 23:13:43.284462 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:43 dist-test-slave-j2vl krb5kdc[13276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:43.429507 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39073
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:42953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:39073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:43.518496 13306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:43.518724 13306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:43.518749 13306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:43.519871 13306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:43.519917 13306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:43.519932 13306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:43.519943 13306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:43.519955 13306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:43.521433 13306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:42953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:39073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:39073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:43.521651 13306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:43.521829 13306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:43.527060 13320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:43.527118 13313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:43.527323 13312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:43.527267 13306 server_base.cc:1034] running on GCE node
I20260809 23:13:43.527580 13306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:43.527884 13306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:43.529062 13306 hybrid_clock.cc:648] HybridClock initialized: now 1786317223529039 us; error 44 us; skew 500 ppm
Aug 09 23:13:43 dist-test-slave-j2vl krb5kdc[13276](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317223, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:43.535902 13306 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:43.536429 13306 webserver.cc:460] Webserver started at http://127.8.141.254:35761/ using document root <none> and password file <none>
I20260809 23:13:43.536695 13306 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:43.536746 13306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:43.536873 13306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:43.538089 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "9e74754177ec434790960737e3f7d2f8"
format_stamp: "Formatted at 2026-08-09 23:13:43 on dist-test-slave-j2vl"
server_key: "039000f70a0378b10a0393f99af15979"
server_key_iv: "5180c46f6440b03f288887d12801c613"
server_key_version: "encryptionkey@0"
I20260809 23:13:43.538468 13306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "9e74754177ec434790960737e3f7d2f8"
format_stamp: "Formatted at 2026-08-09 23:13:43 on dist-test-slave-j2vl"
server_key: "039000f70a0378b10a0393f99af15979"
server_key_iv: "5180c46f6440b03f288887d12801c613"
server_key_version: "encryptionkey@0"
I20260809 23:13:43.540019 13306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:43.541065 13328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:43.541455 13306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:43.541579 13306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "9e74754177ec434790960737e3f7d2f8"
format_stamp: "Formatted at 2026-08-09 23:13:43 on dist-test-slave-j2vl"
server_key: "039000f70a0378b10a0393f99af15979"
server_key_iv: "5180c46f6440b03f288887d12801c613"
server_key_version: "encryptionkey@0"
I20260809 23:13:43.541676 13306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:43.545284 13306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:43.547544 13306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:43.547729 13306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:43.551996 13306 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:39073
I20260809 23:13:43.552028 13409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:39073 every 8 connection(s)
I20260809 23:13:43.552402 13306 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:43.553606 13410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:43.555819 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13306
I20260809 23:13:43.555948 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:43.556071 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8: Bootstrap starting.
I20260809 23:13:43.556141 8759 external_mini_cluster.cc:1428] Setting key 29ba2add2029529b2029b9d3b0db7353
I20260809 23:13:43.556861 13410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:43.557196 13410 log.cc:824] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:43.558010 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8: No bootstrap required, opened a new log
I20260809 23:13:43.559027 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:43.556689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48779 (local address 127.8.141.254:39073)
0809 23:13:43.556896 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:43.556903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:43.556954 (+ 51us) server_negotiation.cc:407] Connection header received
0809 23:13:43.557648 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:43.557657 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:43.557686 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:43.557812 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:43.558529 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:43.558533 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:43.558533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:43.558538 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:43.558578 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:43.558588 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:43.558669 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:43.558737 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 23:13:43.558770 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
I20260809 23:13:43.560355 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:43.560635 13410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e74754177ec434790960737e3f7d2f8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39073 } }
I20260809 23:13:43.560811 13410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 23:13:43.560869 13410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e74754177ec434790960737e3f7d2f8, State: Initialized, Role: FOLLOWER
I20260809 23:13:43.561008 13410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e74754177ec434790960737e3f7d2f8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39073 } }
I20260809 23:13:43.561102 13410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 23:13:43.561139 13410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 23:13:43.561198 13410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:43.562109 13410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e74754177ec434790960737e3f7d2f8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39073 } }
I20260809 23:13:43.562237 13410 leader_election.cc:302] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e74754177ec434790960737e3f7d2f8; no voters:
I20260809 23:13:43.562412 13410 leader_election.cc:288] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 23:13:43.562451 13426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:43.562884 13410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:43.563161 13426 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [term 1 LEADER]: Becoming Leader. State: Replica: 9e74754177ec434790960737e3f7d2f8, State: Running, Role: LEADER
I20260809 23:13:43.563314 13426 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e74754177ec434790960737e3f7d2f8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39073 } }
I20260809 23:13:43.563609 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:43.562093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36457 (local address 127.8.141.254:39073)
0809 23:13:43.562136 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:43.562137 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:43.562233 (+ 96us) server_negotiation.cc:407] Connection header received
0809 23:13:43.562342 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:43.562343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:43.562349 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:43.562394 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:43.563213 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:43.563215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:43.563215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 23:13:43.563219 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:43.563234 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 23:13:43.563238 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:43.563292 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:43.563538 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 23:13:43.563553 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:13:43.563944 13427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e74754177ec434790960737e3f7d2f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e74754177ec434790960737e3f7d2f8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39073 } } }
I20260809 23:13:43.564025 13427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:43.564240 13428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e74754177ec434790960737e3f7d2f8. Latest consensus state: current_term: 1 leader_uuid: "9e74754177ec434790960737e3f7d2f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e74754177ec434790960737e3f7d2f8" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 39073 } } }
I20260809 23:13:43.564548 13428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8 [sys.catalog]: This master's current role is: LEADER
W20260809 23:13:43.564824 13455 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:43.564895 13455 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:43.564949 13448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:43.565095 13448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:43.566650 13448 catalog_manager.cc:1347] Generated new cluster ID: 2f95dcf5141c42c5b8db62056ea64144
I20260809 23:13:43.566707 13448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:43.578497 13448 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:43.578694 13448 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:43.582507 13448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e74754177ec434790960737e3f7d2f8: Generated new TSK 0
I20260809 23:13:43.582720 13448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 23:13:43.585955 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13306
2026-08-09T23:13:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1644 ms)
[----------] 18 tests from AuthTokenIssuingTest (30158 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:43 dist-test-slave-j2vl krb5kdc[13464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:43 dist-test-slave-j2vl krb5kdc[13464](info): set up 2 sockets
Aug 09 23:13:43 dist-test-slave-j2vl krb5kdc[13464](info): commencing operation
krb5kdc: starting...
W20260809 23:13:44.934630 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.304s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:45.077494 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:43847
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:43847,127.8.141.253:46573,127.8.141.252:40105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:45.173974 13490 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:13:45.174283 13490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:45.174316 13490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:45.174363 13490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:45.175669 13490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:45.175763 13490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:45.175787 13490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:45.175810 13490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:45.175839 13490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:45.177567 13490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:43847,127.8.141.253:46573,127.8.141.252:40105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:43847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:45.178308 13490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:45.178689 13490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:45.185683 13496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:45.185681 13500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:45.185832 13498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:45.186017 13490 server_base.cc:1034] running on GCE node
I20260809 23:13:45.186426 13490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:45.186947 13490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:45.188341 13490 hybrid_clock.cc:648] HybridClock initialized: now 1786317225188321 us; error 38 us; skew 500 ppm
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:45.191748 13490 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:45.192338 13490 webserver.cc:460] Webserver started at http://127.8.141.254:35885/ using document root <none> and password file <none>
I20260809 23:13:45.192592 13490 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:45.192667 13490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:45.192834 13490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:45.194087 13490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "7a09886024de42128c94fe7d2cbdf6ff"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "2ddf80ad37ec7d03e04cfc8007d6c8e7"
server_key_iv: "2dfbc513650010c069d0cdff6a4dad98"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.194483 13490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "7a09886024de42128c94fe7d2cbdf6ff"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "2ddf80ad37ec7d03e04cfc8007d6c8e7"
server_key_iv: "2dfbc513650010c069d0cdff6a4dad98"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.195974 13490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:45.197132 13511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:45.197481 13490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:45.197585 13490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "7a09886024de42128c94fe7d2cbdf6ff"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "2ddf80ad37ec7d03e04cfc8007d6c8e7"
server_key_iv: "2dfbc513650010c069d0cdff6a4dad98"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.197695 13490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:45.206678 13490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:45.208884 13490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:45.209019 13490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:45.214183 13490 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:43847
I20260809 23:13:45.214381 13582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:43847 every 8 connection(s)
I20260809 23:13:45.214874 13490 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:45.216068 13584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:45.217872 13584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:45.225206 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13490
I20260809 23:13:45.225368 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:45.225680 8759 external_mini_cluster.cc:1428] Setting key 07f5aa871dc65729ca66d6aa2dfce2cd
I20260809 23:13:45.230126 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.218614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:43847 (local address 127.0.0.1:54810)
0809 23:13:45.218925 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.218979 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.219556 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.219865 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.219868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.220184 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.220354 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.220358 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.221479 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.221484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.222164 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.222166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.222720 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.222722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.222789 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.223496 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.223506 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.225718 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.228041 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.228051 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.228055 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.228368 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.228685 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.228686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.228687 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.228811 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.229422 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.229430 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.229507 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.229597 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.229748 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":129,"threads_started":1}
I20260809 23:13:45.230132 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.218721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.8.141.254:43847)
0809 23:13:45.218937 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.218947 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.219094 (+ 147us) server_negotiation.cc:407] Connection header received
0809 23:13:45.219732 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.219733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.219758 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.219860 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.220466 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.221302 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.222290 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.222601 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.222736 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.225868 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.225883 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.225884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.225892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.227936 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.228511 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.228515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.228515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.228605 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.228904 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.228904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.228905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.229261 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.229422 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.229661 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.229867 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
I20260809 23:13:45.231302 13584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:45.231747 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.231618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:41704)
0809 23:13:45.231672 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.231725 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 23:13:45.231979 13515 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
I20260809 23:13:45.232606 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.232439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:41706)
0809 23:13:45.232551 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.232584 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260809 23:13:45.232770 13584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:46573: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.141.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab.
Entry for principal kudu/127.8.141.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab.
I20260809 23:13:45.261545 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:46573
--webserver_interface=127.8.141.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab
--principal=kudu/127.8.141.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:43847,127.8.141.253:46573,127.8.141.252:40105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:13:45.282056 13584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } attempt: 1
I20260809 23:13:45.282612 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.282461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:59586)
0809 23:13:45.282538 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.282581 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 23:13:45.283103 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.282966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:59594)
0809 23:13:45.283040 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.283078 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 23:13:45.283284 13584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:46573: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
I20260809 23:13:45.351554 13584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } attempt: 2
I20260809 23:13:45.352190 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.352010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:59606)
0809 23:13:45.352105 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.352156 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 23:13:45.352675 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.352547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:59616)
0809 23:13:45.352616 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.352651 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 23:13:45.352561 13597 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:13:45.352829 13597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:45.352878 13597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:45.352845 13584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:46573: Network error: Client connection negotiation failed: client connection to 127.8.141.253:46573: connect: Connection refused (error 111)
W20260809 23:13:45.352919 13597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:45.354337 13597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:45.354414 13597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:45.354440 13597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:45.354472 13597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:45.354508 13597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:45.356523 13597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:43847,127.8.141.253:46573,127.8.141.252:40105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab
--principal=kudu/127.8.141.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:46573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.141.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:45.356889 13597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:45.357172 13597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:45.362772 13610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:45.362841 13604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:45.362854 13606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:45.363251 13597 server_base.cc:1034] running on GCE node
I20260809 23:13:45.363641 13597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:45.363984 13597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:45.365180 13597 hybrid_clock.cc:648] HybridClock initialized: now 1786317225365158 us; error 46 us; skew 500 ppm
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:45.368659 13597 init.cc:377] Logged in from keytab as kudu/127.8.141.253@KRBTEST.COM (short username kudu)
I20260809 23:13:45.369415 13597 webserver.cc:460] Webserver started at http://127.8.141.253:33013/ using document root <none> and password file <none>
I20260809 23:13:45.369693 13597 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:45.369768 13597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:45.369937 13597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:45.371107 13597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data/instance:
uuid: "c7e41f589a804e96b061bb5f00f4f141"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "8a1ba01947d35357b8895255c3ca01d3"
server_key_iv: "090a9635a89d99adbd9396400dbe14d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.371511 13597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/wal/instance:
uuid: "c7e41f589a804e96b061bb5f00f4f141"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "8a1ba01947d35357b8895255c3ca01d3"
server_key_iv: "090a9635a89d99adbd9396400dbe14d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.373026 13597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 23:13:45.373997 13620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:45.374228 13597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 23:13:45.374359 13597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/wal
uuid: "c7e41f589a804e96b061bb5f00f4f141"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "8a1ba01947d35357b8895255c3ca01d3"
server_key_iv: "090a9635a89d99adbd9396400dbe14d4"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.374446 13597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:45.391433 13597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:45.395174 13597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:45.395396 13597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:45.400043 13597 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.253:46573
I20260809 23:13:45.400079 13706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.253:46573 every 8 connection(s)
I20260809 23:13:45.400488 13597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
I20260809 23:13:45.401893 13708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:45.403643 13708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:45.408659 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13597
I20260809 23:13:45.408799 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-1/wal/instance
I20260809 23:13:45.409006 8759 external_mini_cluster.cc:1428] Setting key a0318a336df9797d92a3787fe9e02bf9
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:45.416145 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.404589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40258 (local address 127.8.141.254:43847)
0809 23:13:45.404676 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.404678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.404955 (+ 277us) server_negotiation.cc:407] Connection header received
0809 23:13:45.405534 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.405535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.405550 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.405620 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.406565 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.407095 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.408182 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.408410 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.408474 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.411985 (+ 3511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.411988 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.411988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.411998 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.413908 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.414588 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.414590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.414590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.414660 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.415101 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.415103 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.415104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.415394 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.415473 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.415970 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.416067 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:13:45.416404 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.404567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:43847 (local address 127.0.0.1:40258)
0809 23:13:45.404782 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.404838 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.405412 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.405698 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.405702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.406151 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.406438 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.406444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.407211 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.407221 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.408081 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.408083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.408629 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.408631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.408871 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.409659 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.409671 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.411859 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.414028 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.414043 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.414047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.414473 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.414752 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.414753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.414754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.414937 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.415490 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.415500 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.415721 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.415867 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.416101 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":126,"threads_started":1}
I20260809 23:13:45.417428 13708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.141.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.252@KRBTEST.COM" created.
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:13:45.427426 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.417766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:59622)
0809 23:13:45.417826 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.417839 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.417884 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.418282 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.418283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.418438 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.418505 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.418506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.419690 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.419691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.420263 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.420266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.420822 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.420823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.420892 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.421442 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.421448 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.423490 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.425961 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.425962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.425963 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.426222 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.426612 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.426613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.426613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.426686 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.427145 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.427146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.427190 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.427256 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.427351 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 23:13:45.427610 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.417849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.8.141.253:46573)
0809 23:13:45.418062 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.418069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.418099 (+ 30us) server_negotiation.cc:407] Connection header received
0809 23:13:45.418128 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.418130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.418147 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.418247 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.418596 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.419579 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.420393 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.420726 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.420821 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.423604 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.423618 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.423618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.423627 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.425858 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.426342 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.426346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.426347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.426428 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.426802 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.426803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.426804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.427059 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.427175 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.427326 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.427433 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":119,"threads_started":1}
I20260809 23:13:45.428505 13708 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:45.428970 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.428798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44710)
0809 23:13:45.428866 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.428932 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 23:13:45.429142 13624 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
I20260809 23:13:45.429694 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.429560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44724)
0809 23:13:45.429622 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.429660 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 23:13:45.429852 13708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:40105: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab.
Entry for principal kudu/127.8.141.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab.
I20260809 23:13:45.442729 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:40105
--webserver_interface=127.8.141.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab
--principal=kudu/127.8.141.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:34109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:43847,127.8.141.253:46573,127.8.141.252:40105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:13:45.444029 13584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } attempt: 3
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:13:45.453583 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.444470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:59626)
0809 23:13:45.444616 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.444629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.444690 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.444884 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.444885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.445076 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.445147 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.445149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.446140 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.446141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.446850 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.446860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.447627 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.447629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.447751 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.448541 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.448547 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.450025 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.452318 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.452320 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.452320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.452552 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.452821 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.452822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.452822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.452886 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.453226 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.453227 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.453274 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.453327 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.453506 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260809 23:13:45.453670 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.444550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59626 (local address 127.8.141.253:46573)
0809 23:13:45.444625 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.444629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.444669 (+ 40us) server_negotiation.cc:407] Connection header received
0809 23:13:45.444787 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.444792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.444803 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.444877 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.445240 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.446040 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.447028 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.447498 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.447616 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.450141 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.450144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.450144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.450156 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.452193 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.452647 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.452648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.452649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.452726 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.452961 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.452963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.452963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.453148 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.453236 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.453409 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.453521 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 23:13:45.454146 13584 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:45.454538 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.454425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44728)
0809 23:13:45.454478 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.454517 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 23:13:45.454946 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.454855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44730)
0809 23:13:45.454904 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.454930 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 23:13:45.455085 13584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:40105: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
I20260809 23:13:45.479094 13708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } attempt: 1
I20260809 23:13:45.479712 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.479545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44734)
0809 23:13:45.479622 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.479675 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 23:13:45.480131 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.480023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44744)
0809 23:13:45.480077 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.480110 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 23:13:45.480283 13708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:40105: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
I20260809 23:13:45.486254 13584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } attempt: 1
I20260809 23:13:45.486812 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.486668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44758)
0809 23:13:45.486734 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.486781 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 23:13:45.487202 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.487103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44768)
0809 23:13:45.487150 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.487183 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 23:13:45.487319 13584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:40105: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
W20260809 23:13:45.538225 13720 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:13:45.538529 13720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:45.538561 13720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:45.538609 13720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:45.539878 13720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:45.539966 13720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:45.539991 13720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:45.540014 13720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:45.540045 13720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:45.541710 13720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:34109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:43847,127.8.141.253:46573,127.8.141.252:40105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab
--principal=kudu/127.8.141.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:40105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.141.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:45.542042 13720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:45.542312 13720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:13:45.548450 13708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } attempt: 2
I20260809 23:13:45.549075 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.548811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44770)
0809 23:13:45.548878 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.548906 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 23:13:45.549036 13732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:45.549022 13730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:45.549022 13728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:45.549332 13720 server_base.cc:1034] running on GCE node
I20260809 23:13:45.549692 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.549566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44780)
0809 23:13:45.549623 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.549655 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 23:13:45.549793 13708 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:40105: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
I20260809 23:13:45.549877 13720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:45.550181 13720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:45.551339 13720 hybrid_clock.cc:648] HybridClock initialized: now 1786317225551331 us; error 39 us; skew 500 ppm
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:45.554152 13720 init.cc:377] Logged in from keytab as kudu/127.8.141.252@KRBTEST.COM (short username kudu)
I20260809 23:13:45.554734 13720 webserver.cc:460] Webserver started at http://127.8.141.252:35141/ using document root <none> and password file <none>
I20260809 23:13:45.554984 13720 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:45.555056 13720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:45.555217 13720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:45.556396 13720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data/instance:
uuid: "cd8a3e09b2994e688a84fb835f6f88c1"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "5c7f814112625f9540500f75b07a26c0"
server_key_iv: "c282129d2a4775463249aee95fbe9b7b"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.556787 13720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/wal/instance:
uuid: "cd8a3e09b2994e688a84fb835f6f88c1"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "5c7f814112625f9540500f75b07a26c0"
server_key_iv: "c282129d2a4775463249aee95fbe9b7b"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.558379 13720 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:45.559325 13742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:45.559584 13720 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 23:13:45.559696 13720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/wal
uuid: "cd8a3e09b2994e688a84fb835f6f88c1"
format_stamp: "Formatted at 2026-08-09 23:13:45 on dist-test-slave-j2vl"
server_key: "5c7f814112625f9540500f75b07a26c0"
server_key_iv: "c282129d2a4775463249aee95fbe9b7b"
server_key_version: "encryptionkey@0"
I20260809 23:13:45.559808 13720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:45.562492 13584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } attempt: 2
I20260809 23:13:45.562956 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.562797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44784)
0809 23:13:45.562870 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.562919 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 23:13:45.563476 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.563309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44800)
0809 23:13:45.563380 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.563445 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 23:13:45.563607 13584 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:40105: Network error: Client connection negotiation failed: client connection to 127.8.141.252:40105: connect: Connection refused (error 111)
I20260809 23:13:45.575887 13720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:45.578579 13720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:45.578758 13720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:45.583544 13720 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.252:40105
I20260809 23:13:45.583621 13829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.252:40105 every 8 connection(s)
I20260809 23:13:45.583999 13720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
I20260809 23:13:45.585201 13830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:45.586930 13830 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:45.590077 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 13720
I20260809 23:13:45.590217 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786316926160006-8759-0/minicluster-data/master-2/wal/instance
I20260809 23:13:45.590408 8759 external_mini_cluster.cc:1428] Setting key 7655ab6b384875bf6a7a255f9a500cea
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
I20260809 23:13:45.598338 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.587686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:43847 (local address 127.0.0.1:40268)
0809 23:13:45.587910 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.587966 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.588477 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.588752 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.588755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.589072 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.589289 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.589294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.590120 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.590121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.591409 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.591413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.591850 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.591852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.591985 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.592564 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.592574 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.594291 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.596164 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.596176 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.596179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.596582 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.596864 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.596866 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.596866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.597041 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.597573 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.597585 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.597827 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.597955 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.598107 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":134,"threads_started":1}
I20260809 23:13:45.598361 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.587735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.8.141.254:43847)
0809 23:13:45.587902 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.587904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.588080 (+ 176us) server_negotiation.cc:407] Connection header received
0809 23:13:45.588586 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.588587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.588599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.588668 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.589462 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.590004 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.591524 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.591728 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.591820 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.594399 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.594407 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.594407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.594417 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.596054 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.596714 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.596716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.596716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.596794 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.597150 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.597151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.597151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.597415 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.597520 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.598122 (+ 602us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.598217 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260809 23:13:45.599217 13830 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:45.601176 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.590920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.8.141.252:40105)
0809 23:13:45.591226 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.591231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.591258 (+ 27us) server_negotiation.cc:407] Connection header received
0809 23:13:45.591282 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.591284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.591311 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.591406 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.592059 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.592777 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.593636 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.594061 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.594150 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.597229 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.597259 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.597259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.597268 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.599448 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.600011 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.600015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.600015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.600078 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.600358 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.600360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.600360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.600579 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.600750 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.600832 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.600920 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":215,"threads_started":1}
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:45.607733 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.599540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:46573 (local address 127.0.0.1:59642)
0809 23:13:45.599577 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.599587 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.599619 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.599839 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.599841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.600004 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.600063 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.600065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.601023 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.601025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.601792 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.601796 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.602326 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.602327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.602406 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.602937 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.602943 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.604200 (+ 1257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.606301 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.606302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.606302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.606551 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.606764 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.606765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.606766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.606839 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.607291 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.607292 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.607343 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.607483 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.607617 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 23:13:45.607733 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.599662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59642 (local address 127.8.141.253:46573)
0809 23:13:45.599707 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.599709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.599717 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:13:45.599751 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.599752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.599761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.599801 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.600162 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.600936 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.601878 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.602245 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.602306 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.604308 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.604310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.604310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.604319 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.606180 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.606634 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.606635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.606635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.606693 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.606937 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.606938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.606939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.607097 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.607187 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.607526 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.607617 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:13:45.608913 13830 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:45.610587 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.602222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.8.141.254:43847)
0809 23:13:45.602280 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.602282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.602291 (+ 9us) server_negotiation.cc:407] Connection header received
0809 23:13:45.602317 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.602317 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.602328 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.602388 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.602885 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.603357 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.604149 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.604361 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.604448 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.607382 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.607384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.607384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.607392 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.609165 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.609836 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.609838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.609838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.609895 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.610120 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.610120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.610121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.610225 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.610354 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.610457 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.610522 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:13:45.617177 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.609275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44826 (local address 127.8.141.252:40105)
0809 23:13:45.609321 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.609323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.609333 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:13:45.609416 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.609417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.609431 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.609496 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.609748 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.610571 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.611499 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.611708 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.611784 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.614091 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.614092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.614093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.614100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.615862 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.616237 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.616238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.616239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.616305 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.616553 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.616555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.616555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.616706 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.616792 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.616934 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.617069 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":22}
I20260809 23:13:45.617194 13833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.609242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44826)
0809 23:13:45.609279 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.609288 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.609325 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.609506 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.609507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.609628 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.609666 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.609667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.610664 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.610665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.611404 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.611407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.611779 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.611780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.611829 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.612361 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.612367 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.613980 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.615964 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.615965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.615966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.616156 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.616407 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.616408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.616408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.616474 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.616781 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.616782 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.616825 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.616896 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.617005 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 23:13:45.619238 13830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: Bootstrap starting.
I20260809 23:13:45.619992 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.611416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.8.141.253:46573)
0809 23:13:45.611488 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.611491 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.611499 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:13:45.611523 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.611524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.611534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.611594 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.612292 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.613010 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.613996 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.614261 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.614317 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.617082 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.617084 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.617084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.617093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.618649 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.619055 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.619056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.619057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.619121 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.619442 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.619448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.619448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.619565 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.619764 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.619803 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.619885 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 23:13:45.620296 13830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:45.620641 13830 log.cc:824] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:45.621722 13830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: No bootstrap required, opened a new log
I20260809 23:13:45.621891 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:45.623704 13830 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.623880 13830 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8a3e09b2994e688a84fb835f6f88c1, State: Initialized, Role: FOLLOWER
I20260809 23:13:45.623241 13549 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7a09886024de42128c94fe7d2cbdf6ff"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40284
I20260809 23:13:45.623982 13830 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.624370 13838 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.624444 13830 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:45.624480 13838 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:13:45.626770 13854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:45.627085 13854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:45.627157 13720 master_runner.cc:186] Error getting master registration for 127.8.141.254:43847: OK, instance_id { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" instance_seqno: 1786317225212477 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:13:45.630815 13720 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:13:45.640933 13708 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } attempt: 3
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
I20260809 23:13:45.650079 13712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.641382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44830)
0809 23:13:45.641463 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.641479 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.641551 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.641722 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.641724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.641942 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.642003 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.642005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.642967 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.642968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.644384 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.644388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.644914 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.644915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.644978 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.645523 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.645529 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.646843 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.648815 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.648817 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.648817 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.649064 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.649299 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.649299 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.649300 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.649394 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.649736 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.649736 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.649789 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.649851 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.649964 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 23:13:45.650102 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.641473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44830 (local address 127.8.141.252:40105)
0809 23:13:45.641557 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.641560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.641576 (+ 16us) server_negotiation.cc:407] Connection header received
0809 23:13:45.641621 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.641623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.641634 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.641692 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.642095 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.642864 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.644498 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.644824 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.644905 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.646937 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.646939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.646939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.646948 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.648710 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.649152 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.649153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.649154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.649218 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.649447 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.649448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.649448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.649666 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.649749 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.649913 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.650004 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 23:13:45.651948 13708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: Bootstrap starting.
I20260809 23:13:45.652791 13708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:45.653131 13708 log.cc:824] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:45.654098 13708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: No bootstrap required, opened a new log
I20260809 23:13:45.656347 13708 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.656513 13708 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e41f589a804e96b061bb5f00f4f141, State: Initialized, Role: FOLLOWER
I20260809 23:13:45.656600 13708 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.656994 13708 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:45.656952 13859 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.657056 13859 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:13:45.658981 13879 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:45.659101 13879 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:45.659233 13597 master_runner.cc:186] Error getting master registration for 127.8.141.254:43847: OK, instance_id { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" instance_seqno: 1786317225212477 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:13:45.662473 13597 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:13:45.662792 13584 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } attempt: 3
Aug 09 23:13:45 dist-test-slave-j2vl krb5kdc[13464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317225, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
I20260809 23:13:45.672386 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.663219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44844 (local address 127.8.141.252:40105)
0809 23:13:45.663315 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.663317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.663325 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:13:45.663387 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.663388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.663399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.663445 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.663894 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.664630 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.665883 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.666197 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.666283 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.668728 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.668729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.668756 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.668766 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.670838 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.671402 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.671403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.671404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.671480 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.671751 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.671752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.671753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.671905 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.671978 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.672185 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.672280 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 23:13:45.672406 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.663109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:40105 (local address 127.0.0.1:44844)
0809 23:13:45.663163 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:45.663175 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:45.663214 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:45.663506 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:45.663508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:45.663751 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:45.663810 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.663813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.664770 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.664771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.665781 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:45.665784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.666296 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.666297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:45.666368 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.666925 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:45.666932 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:45.668640 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:45.670949 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.670951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.670952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.671247 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.671564 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:45.671565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:45.671565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:45.671660 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:45.671987 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:45.671988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:45.672062 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 23:13:45.672143 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 23:13:45.672277 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 23:13:45.674943 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff: Bootstrap starting.
I20260809 23:13:45.675781 13584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:45.676122 13584 log.cc:824] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:45.677232 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff: No bootstrap required, opened a new log
I20260809 23:13:45.678900 13584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.679066 13584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a09886024de42128c94fe7d2cbdf6ff, State: Initialized, Role: FOLLOWER
I20260809 23:13:45.679155 13584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.679536 13882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.679591 13549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 23:13:45.679543 13584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:45.679811 13549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:45.680764 13549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.681027 13549 leader_election.cc:288] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [CANDIDATE]: Term 1 election: Requested vote from peers c7e41f589a804e96b061bb5f00f4f141 (127.8.141.253:46573), cd8a3e09b2994e688a84fb835f6f88c1 (127.8.141.252:40105)
W20260809 23:13:45.682345 13896 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:45.682410 13896 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:45.682924 13668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a09886024de42128c94fe7d2cbdf6ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c7e41f589a804e96b061bb5f00f4f141"
I20260809 23:13:45.683071 13668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:45.684092 13668 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a09886024de42128c94fe7d2cbdf6ff in term 1.
I20260809 23:13:45.684373 13787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7a09886024de42128c94fe7d2cbdf6ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cd8a3e09b2994e688a84fb835f6f88c1"
I20260809 23:13:45.684506 13787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:45.685560 13515 leader_election.cc:302] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a09886024de42128c94fe7d2cbdf6ff, c7e41f589a804e96b061bb5f00f4f141; no voters:
I20260809 23:13:45.686546 13787 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a09886024de42128c94fe7d2cbdf6ff in term 1.
I20260809 23:13:45.688716 13490 master_runner.cc:263] No leader master found from master 7a09886024de42128c94fe7d2cbdf6ff
I20260809 23:13:45.688767 13490 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:13:45.679764 13882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:45.688966 13882 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:45.689061 13882 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [term 1 LEADER]: Becoming Leader. State: Replica: 7a09886024de42128c94fe7d2cbdf6ff, State: Running, Role: LEADER
I20260809 23:13:45.689191 13882 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } }
I20260809 23:13:45.689826 13882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a09886024de42128c94fe7d2cbdf6ff. Latest consensus state: current_term: 1 leader_uuid: "7a09886024de42128c94fe7d2cbdf6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.689970 13882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:45.691475 13906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:45.691792 13906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:45.709429 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.690353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.8.141.254:43847)
0809 23:13:45.690398 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.690400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.693585 (+ 3185us) server_negotiation.cc:407] Connection header received
0809 23:13:45.693698 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.693699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.693712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.693742 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.694627 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.695394 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.699725 (+ 4331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.700042 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.700090 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.704135 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.704137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.704137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.704147 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.705872 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.706306 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.706308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.706308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.706370 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.706564 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.706565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.706566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.706703 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.706813 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.709191 (+ 2378us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.709298 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 23:13:45.709928 13519 tablet.cc:2369] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 23:13:45.711501 13714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.690792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.8.141.253:46573)
0809 23:13:45.690834 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.690836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.696587 (+ 5751us) server_negotiation.cc:407] Connection header received
0809 23:13:45.696691 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.696692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.696705 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.696735 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.697703 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.698644 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.701615 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.701951 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.702001 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.707029 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.707032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.707032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.707041 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.708740 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.710581 (+ 1841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.710582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.710583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.710660 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.710882 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.710882 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.710883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.711022 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.711103 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.711310 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.711401 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:13:45.712285 13836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:45.691181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.8.141.252:40105)
0809 23:13:45.691238 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.691240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.697624 (+ 6384us) server_negotiation.cc:407] Connection header received
0809 23:13:45.697668 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.697670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.697684 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.697772 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.705634 (+ 7862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.706397 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.707465 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:45.707776 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:45.707819 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:45.709315 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:45.709317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:45.709318 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:45.709325 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:45.711031 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.711420 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.711422 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.711423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.711484 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:45.711682 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:45.711682 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:45.711683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:45.711822 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:45.711891 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:45.712117 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 23:13:45.712198 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 23:13:45.713625 13787 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [term 1 FOLLOWER]: Refusing update from remote peer 7a09886024de42128c94fe7d2cbdf6ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:45.714040 13904 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:45.715590 13838 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a09886024de42128c94fe7d2cbdf6ff. Latest consensus state: current_term: 1 leader_uuid: "7a09886024de42128c94fe7d2cbdf6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.715694 13838 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:45.716261 13668 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [term 1 FOLLOWER]: Refusing update from remote peer 7a09886024de42128c94fe7d2cbdf6ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:45.716602 13904 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:45.718592 13859 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a09886024de42128c94fe7d2cbdf6ff. Latest consensus state: current_term: 1 leader_uuid: "7a09886024de42128c94fe7d2cbdf6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.718700 13859 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:45.725895 13904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a09886024de42128c94fe7d2cbdf6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.726087 13904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:45.726954 13838 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a09886024de42128c94fe7d2cbdf6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.727051 13838 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:45.728299 13911 mvcc.cc:205] Tried to move back new op lower bound from 7316755356520501248 to 7316755356424216576. Current Snapshot: MvccSnapshot[applied={T|T < 7316755356520501248}]
I20260809 23:13:45.729308 13906 catalog_manager.cc:1347] Generated new cluster ID: 1022c16565814215b6880ed0bf400d5a
I20260809 23:13:45.729439 13906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:45.729677 13882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a09886024de42128c94fe7d2cbdf6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.729768 13882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:45.730227 13859 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a09886024de42128c94fe7d2cbdf6ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a09886024de42128c94fe7d2cbdf6ff" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 43847 } } peers { permanent_uuid: "c7e41f589a804e96b061bb5f00f4f141" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 46573 } } peers { permanent_uuid: "cd8a3e09b2994e688a84fb835f6f88c1" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 40105 } } }
I20260809 23:13:45.730309 13859 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:45.730665 13914 mvcc.cc:205] Tried to move back new op lower bound from 7316755356520501248 to 7316755356424216576. Current Snapshot: MvccSnapshot[applied={T|T < 7316755356520501248}]
I20260809 23:13:45.759155 13906 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:45.760018 13906 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:45.777511 13906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a09886024de42128c94fe7d2cbdf6ff: Generated new TSK 0
I20260809 23:13:45.777923 13906 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:13:45.810843 13588 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.808977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.8.141.254:43847)
0809 23:13:45.809028 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.809031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.810076 (+ 1045us) server_negotiation.cc:407] Connection header received
0809 23:13:45.810190 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.810191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.810204 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.810249 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.810759 (+ 510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40306: BlockingRecv error: recv got EOF from 127.0.0.1:40306 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 23:13:45.812072 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.809436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.8.141.253:46573)
0809 23:13:45.809480 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.809482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.811281 (+ 1799us) server_negotiation.cc:407] Connection header received
0809 23:13:45.811548 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.811549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.811563 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.811596 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.812002 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59668: BlockingRecv error: recv got EOF from 127.0.0.1:59668 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 23:13:45.812920 13836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.809790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.8.141.252:40105)
0809 23:13:45.809853 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.809855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.811492 (+ 1637us) server_negotiation.cc:407] Connection header received
0809 23:13:45.812465 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.812467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.812479 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.812513 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.812854 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44858: BlockingRecv error: recv got EOF from 127.0.0.1:44858 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260809 23:13:45.815943 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.815050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.8.141.253:46573)
0809 23:13:45.815091 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.815093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.815350 (+ 257us) server_negotiation.cc:407] Connection header received
0809 23:13:45.815466 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.815467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.815490 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.815522 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.815889 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59670: BlockingRecv error: recv got EOF from 127.0.0.1:59670 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:13:45.819284 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.818402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.8.141.253:46573)
0809 23:13:45.818440 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.818441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.818716 (+ 275us) server_negotiation.cc:407] Connection header received
0809 23:13:45.818821 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.818822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.818834 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.818864 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.819227 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59680: BlockingRecv error: recv got EOF from 127.0.0.1:59680 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 23:13:45.823628 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.822656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.8.141.253:46573)
0809 23:13:45.822702 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.822704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.822992 (+ 288us) server_negotiation.cc:407] Connection header received
0809 23:13:45.823097 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.823098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.823110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.823141 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.823564 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59686: BlockingRecv error: recv got EOF from 127.0.0.1:59686 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 23:13:45.830221 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.829149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.8.141.253:46573)
0809 23:13:45.829204 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.829207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.829535 (+ 328us) server_negotiation.cc:407] Connection header received
0809 23:13:45.829689 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.829690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.829703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.829732 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.830160 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59692: BlockingRecv error: recv got EOF from 127.0.0.1:59692 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 23:13:45.840881 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.839997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59704 (local address 127.8.141.253:46573)
0809 23:13:45.840056 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.840058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.840172 (+ 114us) server_negotiation.cc:407] Connection header received
0809 23:13:45.840261 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.840262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.840277 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.840362 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.840811 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59704: BlockingRecv error: recv got EOF from 127.0.0.1:59704 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 23:13:45.859238 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.858311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.8.141.253:46573)
0809 23:13:45.858361 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.858363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.858511 (+ 148us) server_negotiation.cc:407] Connection header received
0809 23:13:45.858618 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.858619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.858630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.858659 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.859160 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59708: BlockingRecv error: recv got EOF from 127.0.0.1:59708 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 23:13:45.893730 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.892709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.8.141.253:46573)
0809 23:13:45.892830 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.892832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.892844 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:13:45.892968 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.892969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.892982 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.893046 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.893628 (+ 582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59722: BlockingRecv error: recv got EOF from 127.0.0.1:59722 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 23:13:45.960721 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:45.959784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.8.141.253:46573)
0809 23:13:45.960019 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:45.960021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:45.960031 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:13:45.960059 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:45.960060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:45.960071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:45.960128 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:45.960637 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59732: BlockingRecv error: recv got EOF from 127.0.0.1:59732 (error 108)
Metrics: {"server-negotiator.queue_time_us":201}
W20260809 23:13:46.091099 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:46.090220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.8.141.253:46573)
0809 23:13:46.090306 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:46.090308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:46.090442 (+ 134us) server_negotiation.cc:407] Connection header received
0809 23:13:46.090476 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:46.090477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:46.090489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:46.090553 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:46.091037 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59738: BlockingRecv error: recv got EOF from 127.0.0.1:59738 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 23:13:46.349633 13714 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:46.348680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.8.141.253:46573)
0809 23:13:46.348737 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:46.348740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:46.348797 (+ 57us) server_negotiation.cc:407] Connection header received
0809 23:13:46.348908 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:46.348910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:46.348925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:46.349023 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:46.349554 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59754: BlockingRecv error: recv got EOF from 127.0.0.1:59754 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 23:13:46.627621 13854 catalog_manager.cc:1259] Loaded cluster ID: 1022c16565814215b6880ed0bf400d5a
I20260809 23:13:46.627745 13854 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: loading cluster ID for follower catalog manager: success
I20260809 23:13:46.628770 13854 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: acquiring CA information for follower catalog manager: success
I20260809 23:13:46.629177 13854 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cd8a3e09b2994e688a84fb835f6f88c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:13:46.659662 13879 catalog_manager.cc:1259] Loaded cluster ID: 1022c16565814215b6880ed0bf400d5a
I20260809 23:13:46.659789 13879 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: loading cluster ID for follower catalog manager: success
I20260809 23:13:46.660727 13879 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: acquiring CA information for follower catalog manager: success
I20260809 23:13:46.661114 13879 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c7e41f589a804e96b061bb5f00f4f141: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:13:46.865728 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:46.862798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.8.141.253:46573)
0809 23:13:46.862937 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:46.862940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:46.863017 (+ 77us) server_negotiation.cc:407] Connection header received
0809 23:13:46.863106 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:46.863107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:46.863168 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:46.863280 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:46.863460 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.864017 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.864811 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.865032 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.865108 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:46.865293 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:46.865471 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:46.865563 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:46.865599 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 23:13:46.865621 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260809 23:13:46.870658 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:46.867684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44866 (local address 127.8.141.252:40105)
0809 23:13:46.867824 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:46.867827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:46.867844 (+ 17us) server_negotiation.cc:407] Connection header received
0809 23:13:46.867913 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:46.867914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:46.867974 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:46.868067 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:46.868201 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.868711 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.869663 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.869927 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.870024 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:46.870238 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:46.870377 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:46.870489 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:46.870515 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 23:13:46.870544 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":68,"threads_started":1}
I20260809 23:13:46.876060 14049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:46.873053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44876 (local address 127.8.141.252:40105)
0809 23:13:46.873095 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:46.873098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:46.873106 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:13:46.873159 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:46.873160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:46.873172 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:46.873237 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:46.873409 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.873912 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.875103 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.875326 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.875379 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:46.875572 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:46.875654 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:46.875751 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:46.875917 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 23:13:46.875950 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:13:46.876948 14064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:46.872769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.8.141.254:43847)
0809 23:13:46.872887 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:46.872889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:46.873062 (+ 173us) server_negotiation.cc:407] Connection header received
0809 23:13:46.873119 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:46.873120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:46.873179 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:46.873257 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:46.873507 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.874328 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.875445 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.875897 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.875978 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:46.876564 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:46.876692 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:46.876759 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:46.876825 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 23:13:46.876855 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":96,"threads_started":1}
I20260809 23:13:46.877041 14043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:46.872728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.8.141.253:46573)
0809 23:13:46.872778 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:46.872781 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:46.872919 (+ 138us) server_negotiation.cc:407] Connection header received
0809 23:13:46.873047 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:46.873048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:46.873065 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:46.873100 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:46.873300 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.874132 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.875448 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:46.875895 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:46.875978 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:46.876674 (+ 696us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:46.876754 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:46.876809 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:46.876921 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 23:13:46.876950 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:13:46.883668 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13490
I20260809 23:13:46.899860 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13597
I20260809 23:13:46.917758 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 13720
2026-08-09T23:13:46Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3325 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 23:13:47 dist-test-slave-j2vl krb5kdc[14069](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 23:13:47 dist-test-slave-j2vl krb5kdc[14069](info): set up 2 sockets
Aug 09 23:13:47 dist-test-slave-j2vl krb5kdc[14069](info): commencing operation
krb5kdc: starting...
W20260809 23:13:48.268189 8759 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/test-user.keytab.
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T23:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T23:13:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.141.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
Entry for principal kudu/127.8.141.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab.
I20260809 23:13:48.409556 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:38349
--webserver_interface=127.8.141.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:38349,127.8.141.253:40013,127.8.141.252:37719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 23:13:48.500372 14089 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:13:48.500661 14089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:48.500694 14089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:48.500751 14089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:48.502058 14089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:48.502137 14089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:48.502163 14089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:48.502194 14089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:48.502249 14089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:48.503914 14089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:38349,127.8.141.253:40013,127.8.141.252:37719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.254.keytab
--principal=kudu/127.8.141.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.254:38349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.141.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:48.504235 14089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:48.504488 14089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:48.510129 14097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:48.510164 14110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:48.510129 14104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:48.510493 14089 server_base.cc:1034] running on GCE node
I20260809 23:13:48.510757 14089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:48.511096 14089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:48.512274 14089 hybrid_clock.cc:648] HybridClock initialized: now 1786317228512259 us; error 38 us; skew 500 ppm
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:48.515245 14089 init.cc:377] Logged in from keytab as kudu/127.8.141.254@KRBTEST.COM (short username kudu)
I20260809 23:13:48.515808 14089 webserver.cc:460] Webserver started at http://127.8.141.254:42919/ using document root <none> and password file <none>
I20260809 23:13:48.516165 14089 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:48.516317 14089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:48.516590 14089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:48.517876 14089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data/instance:
uuid: "2e33b1af2ed14571b0b88679120f94df"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "6b73bc4c0e5501442a2f774a6497eee1"
server_key_iv: "39334ca0e1a24ef9901424ef5d24c0ca"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.518285 14089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/wal/instance:
uuid: "2e33b1af2ed14571b0b88679120f94df"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "6b73bc4c0e5501442a2f774a6497eee1"
server_key_iv: "39334ca0e1a24ef9901424ef5d24c0ca"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.519830 14089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 23:13:48.520905 14127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:48.521229 14089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:48.521354 14089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/wal
uuid: "2e33b1af2ed14571b0b88679120f94df"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "6b73bc4c0e5501442a2f774a6497eee1"
server_key_iv: "39334ca0e1a24ef9901424ef5d24c0ca"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.521477 14089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:48.542335 14089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:48.544867 14089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:48.545053 14089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:48.549504 14089 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.254:38349
I20260809 23:13:48.549574 14217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.254:38349 every 8 connection(s)
I20260809 23:13:48.549952 14089 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/data/info.pb
I20260809 23:13:48.551082 14219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:48.552815 14219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:48.556367 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 14089
I20260809 23:13:48.556512 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-0/wal/instance
I20260809 23:13:48.556706 8759 external_mini_cluster.cc:1428] Setting key 41599666247f2b6e00055d604ebdc4cb
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:48.564242 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.553598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38349 (local address 127.0.0.1:60546)
0809 23:13:48.553798 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.553961 (+ 163us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.554477 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.554802 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.554805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.555149 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.555311 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.555316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.556397 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.556398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.557224 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.557226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.557817 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.557818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.557885 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.558637 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.558652 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.560625 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.562660 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.562667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.562670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.562891 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.563139 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.563140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.563140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.563227 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.563603 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.563609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.563675 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.563786 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.564040 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":98,"thread_start_us":84,"threads_started":1}
I20260809 23:13:48.564240 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.553787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.8.141.254:38349)
0809 23:13:48.554012 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.554017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.554051 (+ 34us) server_negotiation.cc:407] Connection header received
0809 23:13:48.554649 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.554650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.554669 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.554792 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.555422 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.556294 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.557315 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.557669 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.557809 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.560731 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.560740 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.560740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.560750 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.562559 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.562977 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.562980 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.562981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.563067 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.563295 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.563296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.563296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.563531 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.563627 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.563821 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.563971 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":48,"threads_started":1}
I20260809 23:13:48.565070 14219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:48.565521 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.565377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54678)
0809 23:13:48.565442 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.565496 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 23:13:48.565668 14132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.141.253@KRBTEST.COM; defaulting to no policy
I20260809 23:13:48.566232 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.566132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54690)
0809 23:13:48.566181 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.566206 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:13:48.566349 14219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:40013: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab.
Entry for principal kudu/127.8.141.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab.
I20260809 23:13:48.589075 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:40013
--webserver_interface=127.8.141.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab
--principal=kudu/127.8.141.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:38349,127.8.141.253:40013,127.8.141.252:37719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:13:48.615589 14219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } attempt: 1
I20260809 23:13:48.616133 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.615990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54702)
0809 23:13:48.616057 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.616101 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 23:13:48.616487 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.616390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54706)
0809 23:13:48.616436 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.616461 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:13:48.616580 14219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:40013: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
W20260809 23:13:48.679289 14227 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:13:48.679587 14227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:48.679620 14227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:48.679661 14227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:48.680943 14227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:48.681017 14227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:48.681042 14227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:48.681066 14227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:48.681092 14227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:48.683233 14227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:38349,127.8.141.253:40013,127.8.141.252:37719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.253.keytab
--principal=kudu/127.8.141.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.253:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.141.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:48.683602 14227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:48.683951 14227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 23:13:48.684799 14219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } attempt: 2
I20260809 23:13:48.685307 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.685169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54720)
0809 23:13:48.685235 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.685276 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 23:13:48.685694 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.685601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54732)
0809 23:13:48.685638 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.685673 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 23:13:48.685793 14219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.253:40013: Network error: Client connection negotiation failed: client connection to 127.8.141.253:40013: connect: Connection refused (error 111)
W20260809 23:13:48.689592 14241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:48.689553 14235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:48.689553 14238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:48.689728 14227 server_base.cc:1034] running on GCE node
I20260809 23:13:48.690058 14227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:48.690394 14227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:48.691581 14227 hybrid_clock.cc:648] HybridClock initialized: now 1786317228691572 us; error 38 us; skew 500 ppm
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:48.694572 14227 init.cc:377] Logged in from keytab as kudu/127.8.141.253@KRBTEST.COM (short username kudu)
I20260809 23:13:48.695124 14227 webserver.cc:460] Webserver started at http://127.8.141.253:45535/ using document root <none> and password file <none>
I20260809 23:13:48.695325 14227 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:48.695369 14227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:48.695468 14227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:48.696745 14227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data/instance:
uuid: "da4a111058f740dda534844438e25abf"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "b01a84ae3d14e604aa09408d51e578dd"
server_key_iv: "09aedcf423500058a0a036bc11b9f9d9"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.697075 14227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/wal/instance:
uuid: "da4a111058f740dda534844438e25abf"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "b01a84ae3d14e604aa09408d51e578dd"
server_key_iv: "09aedcf423500058a0a036bc11b9f9d9"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.698621 14227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 23:13:48.699515 14257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:48.699781 14227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:48.699890 14227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/wal
uuid: "da4a111058f740dda534844438e25abf"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "b01a84ae3d14e604aa09408d51e578dd"
server_key_iv: "09aedcf423500058a0a036bc11b9f9d9"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.699963 14227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:48.716207 14227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:48.719018 14227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:48.719156 14227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:48.723809 14227 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.253:40013
I20260809 23:13:48.723907 14329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.253:40013 every 8 connection(s)
I20260809 23:13:48.724545 14227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/data/info.pb
I20260809 23:13:48.726121 14330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:48.726341 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 14227
I20260809 23:13:48.726485 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-1/wal/instance
I20260809 23:13:48.726699 8759 external_mini_cluster.cc:1428] Setting key 9a30ae84173ecc2e80236aa77bcf52f7
I20260809 23:13:48.728001 14330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.141.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.141.252@KRBTEST.COM" created.
I20260809 23:13:48.742408 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.728858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.8.141.254:38349)
0809 23:13:48.728946 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.728949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.729235 (+ 286us) server_negotiation.cc:407] Connection header received
0809 23:13:48.730169 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.730171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.730188 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.730252 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.731314 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.731838 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.734307 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.734557 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.734653 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.738250 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.738252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.738253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.738263 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.739959 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.740721 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.740723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.740723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.740831 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.741213 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.741214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.741214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.741484 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.741553 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.742114 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.742232 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:13:48.742426 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.728814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38349 (local address 127.0.0.1:60550)
0809 23:13:48.729048 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.729116 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.729995 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.730335 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.730340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.730834 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.731170 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.731177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.731970 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.731972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.734100 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.734111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.734787 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.734789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.735050 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.736102 (+ 1052us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.736129 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.738101 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.740063 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.740073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.740075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.740348 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.740953 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.740954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.740954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.741081 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.741586 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.741594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.741921 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.742014 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.742227 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":117,"threads_started":1}
I20260809 23:13:48.745193 14330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:13:48.754071 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.745632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54742)
0809 23:13:48.745707 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.745720 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.745810 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.746078 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.746079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.746270 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.746343 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.746345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.747256 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.747258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.747788 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.747790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.748336 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.748337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.748405 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.748901 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.748913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.750470 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.752444 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.752445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.752445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.752710 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.753076 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.753077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.753077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.753142 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.753607 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.753608 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.753655 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.753735 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.753827 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 23:13:48.754083 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.745733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54742 (local address 127.8.141.253:40013)
0809 23:13:48.745934 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.745939 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.745959 (+ 20us) server_negotiation.cc:407] Connection header received
0809 23:13:48.745977 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.745979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.745991 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.746078 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.746437 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.747156 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.747895 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.748232 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.748336 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.750604 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.750615 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.750615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.750623 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.752347 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.752813 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.752816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.752817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.752873 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.753239 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.753241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.753241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.753505 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.753589 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.753814 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.754002 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":107,"threads_started":1}
I20260809 23:13:48.754923 14330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:48.755524 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.755240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:47982)
0809 23:13:48.755420 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.755486 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260809 23:13:48.755707 14259 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
I20260809 23:13:48.756186 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.756080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:47994)
0809 23:13:48.756129 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.756155 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 23:13:48.756374 14330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37719: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.141.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab.
Entry for principal kudu/127.8.141.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab.
I20260809 23:13:48.765532 8759 external_mini_cluster.cc:1326] Running /tmp/dist-test-task1V3VFd/build/release/bin/kudu
/tmp/dist-test-task1V3VFd/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:37719
--webserver_interface=127.8.141.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab
--principal=kudu/127.8.141.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.141.212:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.141.254:38349,127.8.141.253:40013,127.8.141.252:37719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 23:13:48.776970 14219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } attempt: 3
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:13:48.785307 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.777570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54750)
0809 23:13:48.777687 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.777699 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.777790 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.777948 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.777949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.778110 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.778182 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.778184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.779164 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.779165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.779804 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.779808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.780330 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.780331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.780496 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.781100 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.781113 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.782466 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.784164 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.784166 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.784166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.784352 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.784598 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.784599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.784599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.784669 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.784948 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.784950 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.785000 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.785089 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.785201 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 23:13:48.785307 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.777664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.8.141.253:40013)
0809 23:13:48.777734 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.777737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.777774 (+ 37us) server_negotiation.cc:407] Connection header received
0809 23:13:48.777859 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.777860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.777873 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.777931 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.778259 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.779057 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.779897 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.780247 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.780318 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.782568 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.782570 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.782570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.782579 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.784051 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.784461 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.784463 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.784463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.784525 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.784752 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.784753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.784753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.784875 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.784957 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.785116 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.785203 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 23:13:48.785844 14219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } has no permanent_uuid. Determining permanent_uuid...
I20260809 23:13:48.786286 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.786162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48008)
0809 23:13:48.786216 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.786247 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 23:13:48.786732 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.786629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48024)
0809 23:13:48.786677 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.786704 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 23:13:48.786888 14219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37719: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
I20260809 23:13:48.805666 14330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } attempt: 1
I20260809 23:13:48.806236 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.806095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48030)
0809 23:13:48.806162 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.806203 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 23:13:48.806594 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.806496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48040)
0809 23:13:48.806541 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.806569 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 23:13:48.806680 14330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37719: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
I20260809 23:13:48.818094 14219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } attempt: 1
I20260809 23:13:48.818668 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.818523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48052)
0809 23:13:48.818591 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.818635 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 23:13:48.819136 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.819015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48058)
0809 23:13:48.819082 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.819113 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 23:13:48.819437 14219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37719: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
W20260809 23:13:48.858758 14349 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 23:13:48.859002 14349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 23:13:48.859021 14349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 23:13:48.859045 14349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 23:13:48.860363 14349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 23:13:48.860419 14349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 23:13:48.860433 14349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 23:13:48.860447 14349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 23:13:48.860459 14349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 23:13:48.862188 14349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.141.212:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.141.254:38349,127.8.141.253:40013,127.8.141.252:37719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/krb5kdc/kudu_127.8.141.252.keytab
--principal=kudu/127.8.141.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.141.252:37719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.141.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 22:16:40 UTC on ccycloud.quasar-nungeg.root.comops.site
build id 4576814
I20260809 23:13:48.862491 14349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 23:13:48.862778 14349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 23:13:48.868291 14359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:48.868294 14362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 23:13:48.868278 14367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 23:13:48.868602 14349 server_base.cc:1034] running on GCE node
I20260809 23:13:48.868918 14349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 23:13:48.869257 14349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 23:13:48.870421 14349 hybrid_clock.cc:648] HybridClock initialized: now 1786317228870409 us; error 32 us; skew 500 ppm
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 23:13:48.873208 14349 init.cc:377] Logged in from keytab as kudu/127.8.141.252@KRBTEST.COM (short username kudu)
I20260809 23:13:48.873735 14349 webserver.cc:460] Webserver started at http://127.8.141.252:33627/ using document root <none> and password file <none>
I20260809 23:13:48.874007 14349 fs_manager.cc:359] Metadata directory not provided
I20260809 23:13:48.874089 14349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 23:13:48.874250 14349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 23:13:48.874823 14330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } attempt: 2
I20260809 23:13:48.875309 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.875160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48072)
0809 23:13:48.875231 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.875273 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 23:13:48.875764 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.875663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48088)
0809 23:13:48.875712 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.875744 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:13:48.875607 14349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data/instance:
uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "f5504fe96e49e8d206f6f558ad3d7c64"
server_key_iv: "8de9466a8173fa374ea4f50d02db2141"
server_key_version: "encryptionkey@0"
W20260809 23:13:48.875874 14330 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37719: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
I20260809 23:13:48.876053 14349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/wal/instance:
uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "f5504fe96e49e8d206f6f558ad3d7c64"
server_key_iv: "8de9466a8173fa374ea4f50d02db2141"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.877625 14349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 23:13:48.878614 14374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 23:13:48.878836 14349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 23:13:48.878945 14349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data,/tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/wal
uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7"
format_stamp: "Formatted at 2026-08-09 23:13:48 on dist-test-slave-j2vl"
server_key: "f5504fe96e49e8d206f6f558ad3d7c64"
server_key_iv: "8de9466a8173fa374ea4f50d02db2141"
server_key_version: "encryptionkey@0"
I20260809 23:13:48.879001 14349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 23:13:48.894603 14219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } attempt: 2
I20260809 23:13:48.894783 14349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 23:13:48.895107 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.894968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48098)
0809 23:13:48.895035 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.895076 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 23:13:48.895591 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.895488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48110)
0809 23:13:48.895537 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.895569 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 23:13:48.895720 14219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.141.252:37719: Network error: Client connection negotiation failed: client connection to 127.8.141.252:37719: connect: Connection refused (error 111)
I20260809 23:13:48.897624 14349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 23:13:48.897804 14349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 23:13:48.902136 14349 rpc_server.cc:307] RPC server started. Bound to: 127.8.141.252:37719
I20260809 23:13:48.902211 14457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.141.252:37719 every 8 connection(s)
I20260809 23:13:48.902573 14349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/data/info.pb
I20260809 23:13:48.903734 14459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 23:13:48.905405 14459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:48.912365 8759 external_mini_cluster.cc:1388] Started /tmp/dist-test-task1V3VFd/build/release/bin/kudu as pid 14349
I20260809 23:13:48.912505 8759 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task1V3VFd/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786316926160006-8759-0/minicluster-data/master-2/wal/instance
I20260809 23:13:48.912719 8759 external_mini_cluster.cc:1428] Setting key df7a65c34463c2f82cdcdf728717564e
I20260809 23:13:48.914916 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.906157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60554 (local address 127.8.141.254:38349)
0809 23:13:48.906232 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.906234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.906433 (+ 199us) server_negotiation.cc:407] Connection header received
0809 23:13:48.906871 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.906872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.906885 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.906952 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.907602 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.908094 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.909043 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.909228 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.909285 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.911491 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.911492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.911493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.911499 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.913253 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.913812 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.913813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.913813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.913881 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.914201 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.914202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.914202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.914362 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.914447 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.914706 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.914828 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 23:13:48.915009 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.906090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.254:38349 (local address 127.0.0.1:60554)
0809 23:13:48.906316 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.906346 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.906785 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.907031 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.907034 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.907283 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.907518 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.907523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.908173 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.908174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.908960 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.908963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.909322 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.909323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.909489 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.909949 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.909957 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.911376 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.913326 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.913331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.913333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.913702 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.913959 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.913960 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.913960 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.914116 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.914442 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.914448 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.914570 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.914657 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.914788 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
I20260809 23:13:48.915683 14459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:13:48.922358 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.913133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.8.141.252:37719)
0809 23:13:48.913363 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.913369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.913439 (+ 70us) server_negotiation.cc:407] Connection header received
0809 23:13:48.913497 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.913499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.913510 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.913570 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.914361 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.915387 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.916319 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.916648 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.916733 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.919247 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.919256 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.919256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.919264 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.920939 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.921370 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.921374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.921374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.921440 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.921704 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.921706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.921706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.921865 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.921982 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.922122 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.922238 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260809 23:13:48.923337 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.915964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.253:40013 (local address 127.0.0.1:54762)
0809 23:13:48.916015 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.916025 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.916060 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.916261 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.916262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.916400 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.916446 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.916448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.917165 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.917166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.918056 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.918059 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.918472 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.918473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.918556 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.918961 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.918966 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.920280 (+ 1314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.922191 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.922192 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.922192 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.922420 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.922627 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.922628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.922628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.922692 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.922993 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.922994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.923040 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.923130 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.923228 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 23:13:48.923386 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.916048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54762 (local address 127.8.141.253:40013)
0809 23:13:48.916125 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.916128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.916136 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:13:48.916160 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.916161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.916171 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.916242 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.916525 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.917045 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.918164 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.918367 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.918470 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.920381 (+ 1911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.920382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.920383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.920390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.922107 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.922486 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.922487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.922488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.922551 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.922767 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.922768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.922768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.922918 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.922988 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.923160 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.923271 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 23:13:48.923745 14459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.252@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.254@KRBTEST.COM
I20260809 23:13:48.930685 14461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.923996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48134)
0809 23:13:48.924094 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.924105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.924147 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.924258 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.924259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.924364 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.924391 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.924392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.925002 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.925003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.925535 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.925536 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.925908 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.925909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.925956 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.926474 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.926478 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.927650 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.929458 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.929459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.929460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.929684 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.929957 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.929958 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.929958 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.930029 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.930298 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.930298 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.930339 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.930434 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.930538 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 23:13:48.930685 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.924057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48134 (local address 127.8.141.252:37719)
0809 23:13:48.924095 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.924097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.924141 (+ 44us) server_negotiation.cc:407] Connection header received
0809 23:13:48.924192 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.924193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.924203 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.924255 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.924463 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.924911 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.925611 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.925824 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.925907 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.927746 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.927747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.927748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.927755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.929351 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.929787 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.929788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.929788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.929849 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.930099 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.930099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.930100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.930204 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.930307 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.930457 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.930584 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":19}
I20260809 23:13:48.931432 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.923298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60556 (local address 127.8.141.254:38349)
0809 23:13:48.923336 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.923338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.923370 (+ 32us) server_negotiation.cc:407] Connection header received
0809 23:13:48.923391 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.923392 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.923402 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.923450 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.924163 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.924807 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.925580 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.925785 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.925868 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.928408 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.928409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.928410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.928417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.930189 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.930528 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.930529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.930530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.930587 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.930891 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.930892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.930892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.931005 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.931131 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.931256 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.931343 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 23:13:48.933234 14459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: Bootstrap starting.
I20260809 23:13:48.934300 14459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:48.934629 14459 log.cc:824] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:48.935598 14459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: No bootstrap required, opened a new log
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.141.253@KRBTEST.COM
I20260809 23:13:48.937601 14459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:48.937768 14459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7be0cd7a4dea4a029c8aadaf2875b9d7, State: Initialized, Role: FOLLOWER
I20260809 23:13:48.937860 14459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:48.938192 14467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:48.938262 14459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:48.938299 14467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:13:48.940258 14485 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:48.940320 14485 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:48.940948 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.932300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54768 (local address 127.8.141.253:40013)
0809 23:13:48.932365 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.932367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.932398 (+ 31us) server_negotiation.cc:407] Connection header received
0809 23:13:48.932419 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.932419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.932428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.932466 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.933038 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.933565 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.934520 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.934761 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.934815 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.937430 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.937432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.937433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.937441 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.939436 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.939918 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.939919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.939920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.939982 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.940206 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.940207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.940207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.940347 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.940490 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.940762 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.940839 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 23:13:48.942344 8759 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 23:13:48.943200 14171 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2e33b1af2ed14571b0b88679120f94df"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60556
I20260809 23:13:48.941040 14349 master_runner.cc:186] Error getting master registration for 127.8.141.254:38349: OK, instance_id { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" instance_seqno: 1786317228548220 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:13:48.944082 14349 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:13:48.967034 14330 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } attempt: 3
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.253@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
I20260809 23:13:48.976646 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.967540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.8.141.252:37719)
0809 23:13:48.967625 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.967628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.967678 (+ 50us) server_negotiation.cc:407] Connection header received
0809 23:13:48.967709 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.967710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.967730 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.967788 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.968304 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.969074 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.970578 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.970899 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.970977 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.973293 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.973295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.973295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.973304 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:48.975185 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.975706 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.975707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.975708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.975779 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:48.975992 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:48.975993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:48.975993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:48.976214 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:48.976291 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:48.976455 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 23:13:48.976543 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:13:48.976734 14339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.967458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48142)
0809 23:13:48.967552 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.967566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.967620 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.967850 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.967851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.968130 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.968198 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.968200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.969175 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.969176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.970465 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.970468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.970986 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.970987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.971046 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.971588 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.971593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.973198 (+ 1605us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:48.975308 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.975310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.975311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.975617 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.975851 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:48.975852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:48.975852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:48.975922 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:48.976288 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:48.976289 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:48.976343 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 23:13:48.976412 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 23:13:48.976520 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 23:13:48.978488 14330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: Bootstrap starting.
I20260809 23:13:48.979279 14330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:48.979609 14330 log.cc:824] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:48.980458 14330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: No bootstrap required, opened a new log
I20260809 23:13:48.982704 14330 raft_consensus.cc:348] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:48.982867 14330 raft_consensus.cc:729] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da4a111058f740dda534844438e25abf, State: Initialized, Role: FOLLOWER
I20260809 23:13:48.982944 14330 consensus_queue.cc:260] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:48.983218 14489 sys_catalog.cc:455] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:48.983289 14330 sys_catalog.cc:564] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:48.983342 14489 sys_catalog.cc:458] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [sys.catalog]: This master's current role is: FOLLOWER
W20260809 23:13:48.985400 14510 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:48.985461 14510 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:48.985656 14227 master_runner.cc:186] Error getting master registration for 127.8.141.254:38349: OK, instance_id { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" instance_seqno: 1786317228548220 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 23:13:48.988579 14227 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:13:48.994858 14219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } attempt: 3
Aug 09 23:13:48 dist-test-slave-j2vl krb5kdc[14069](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786317228, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.141.254@KRBTEST.COM for kudu/127.8.141.252@KRBTEST.COM
I20260809 23:13:49.002624 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.995415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48154 (local address 127.8.141.252:37719)
0809 23:13:48.995486 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:48.995488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:48.995496 (+ 8us) server_negotiation.cc:407] Connection header received
0809 23:13:48.995518 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:48.995519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:48.995527 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:48.995598 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:48.995967 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.996453 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.997446 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.997638 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.997719 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.999886 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:48.999888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:48.999888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:48.999897 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:49.001309 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.001760 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.001761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.001762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.001821 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.002014 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.002015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.002015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.002191 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:49.002259 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:49.002457 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 23:13:49.002531 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 23:13:49.002624 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:48.995222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.141.252:37719 (local address 127.0.0.1:48154)
0809 23:13:48.995281 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 23:13:48.995292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 23:13:48.995334 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 23:13:48.995609 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 23:13:48.995610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 23:13:48.995819 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 23:13:48.995887 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.995890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.996545 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.996547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.997375 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 23:13:48.997378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 23:13:48.997718 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 23:13:48.997719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 23:13:48.997781 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:48.998279 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 23:13:48.998285 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 23:13:48.999808 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 23:13:49.001468 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:49.001470 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:49.001470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:49.001698 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:49.001895 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 23:13:49.001896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 23:13:49.001896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 23:13:49.001962 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 23:13:49.002249 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 23:13:49.002250 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 23:13:49.002310 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 23:13:49.002402 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 23:13:49.002515 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 23:13:49.004550 14219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df: Bootstrap starting.
I20260809 23:13:49.005218 14219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df: Neither blocks nor log segments found. Creating new log.
I20260809 23:13:49.005633 14219 log.cc:824] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df: Log is configured to *not* fsync() on all Append() calls
I20260809 23:13:49.006425 14219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df: No bootstrap required, opened a new log
I20260809 23:13:49.007467 14219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:49.007606 14219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e33b1af2ed14571b0b88679120f94df, State: Initialized, Role: FOLLOWER
I20260809 23:13:49.007699 14219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:49.007978 14515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.008088 14171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 23:13:49.008036 14219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: configured and running, proceeding with master startup.
I20260809 23:13:49.008113 14515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:49.008145 14171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:49.010253 14171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:49.010541 14171 leader_election.cc:288] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [CANDIDATE]: Term 1 election: Requested vote from peers da4a111058f740dda534844438e25abf (127.8.141.253:40013), 7be0cd7a4dea4a029c8aadaf2875b9d7 (127.8.141.252:37719)
I20260809 23:13:49.010972 14295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2e33b1af2ed14571b0b88679120f94df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da4a111058f740dda534844438e25abf"
I20260809 23:13:49.011127 14295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:49.012148 14295 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e33b1af2ed14571b0b88679120f94df in term 1.
W20260809 23:13:49.012475 14536 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 23:13:49.012537 14536 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 23:13:49.012622 14421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2e33b1af2ed14571b0b88679120f94df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7"
I20260809 23:13:49.012746 14421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 23:13:49.014303 14421 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e33b1af2ed14571b0b88679120f94df in term 1.
I20260809 23:13:49.014776 14132 leader_election.cc:302] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e33b1af2ed14571b0b88679120f94df, 7be0cd7a4dea4a029c8aadaf2875b9d7; no voters:
I20260809 23:13:49.015125 14515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 23:13:49.015281 14515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [term 1 LEADER]: Becoming Leader. State: Replica: 2e33b1af2ed14571b0b88679120f94df, State: Running, Role: LEADER
I20260809 23:13:49.015363 14515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } }
I20260809 23:13:49.015767 14515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e33b1af2ed14571b0b88679120f94df. Latest consensus state: current_term: 1 leader_uuid: "2e33b1af2ed14571b0b88679120f94df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.015852 14515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:49.018102 14089 master_runner.cc:263] No leader master found from master 2e33b1af2ed14571b0b88679120f94df
I20260809 23:13:49.018154 14089 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 23:13:49.018519 14548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 23:13:49.019311 14548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 23:13:49.025799 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:49.012811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54770 (local address 127.8.141.253:40013)
0809 23:13:49.012857 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.012859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.013750 (+ 891us) server_negotiation.cc:407] Connection header received
0809 23:13:49.013855 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.013856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.013868 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.013899 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.014781 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:49.015648 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:49.017782 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:49.018062 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:49.018107 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:49.021240 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:49.021242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:49.021242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:49.021250 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:49.024018 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.024726 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.024728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.024729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.024790 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.025013 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.025014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.025014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.025162 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:49.025241 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:49.025580 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 23:13:49.025678 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:13:49.026911 14464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:49.013186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48164 (local address 127.8.141.252:37719)
0809 23:13:49.013231 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.013233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.016788 (+ 3555us) server_negotiation.cc:407] Connection header received
0809 23:13:49.016884 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.016885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.016896 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.016926 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.019819 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:49.020442 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:49.021190 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:49.021704 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:49.021791 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:49.023333 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:49.023335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:49.023335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:49.023343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:49.025171 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.025686 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.025688 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.025688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.025754 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.026294 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.026295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.026296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.026433 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:49.026519 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:49.026721 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 23:13:49.026807 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 23:13:49.029336 14421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [term 1 FOLLOWER]: Refusing update from remote peer 2e33b1af2ed14571b0b88679120f94df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:49.029877 14295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [term 1 FOLLOWER]: Refusing update from remote peer 2e33b1af2ed14571b0b88679120f94df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 23:13:49.030478 14515 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [LEADER]: Connected to new peer: Peer: permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:49.030674 14515 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 23:13:49.032449 14489 sys_catalog.cc:455] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e33b1af2ed14571b0b88679120f94df. Latest consensus state: current_term: 1 leader_uuid: "2e33b1af2ed14571b0b88679120f94df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.032560 14489 sys_catalog.cc:458] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:49.033514 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:49.013600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.8.141.254:38349)
0809 23:13:49.013645 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.013648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.025506 (+ 11858us) server_negotiation.cc:407] Connection header received
0809 23:13:49.025618 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.025619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.025630 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.025660 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.026343 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:49.027059 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:49.028059 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:49.028247 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:49.028336 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:49.030198 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 23:13:49.030200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 23:13:49.030201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 23:13:49.030210 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 23:13:49.032036 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.032501 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.032502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.032503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.032563 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 23:13:49.032790 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 23:13:49.032791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 23:13:49.032791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 23:13:49.032935 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 23:13:49.033024 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:49.033258 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 23:13:49.033392 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 23:13:49.033831 14467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e33b1af2ed14571b0b88679120f94df. Latest consensus state: current_term: 1 leader_uuid: "2e33b1af2ed14571b0b88679120f94df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.033937 14467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:49.035632 14544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e33b1af2ed14571b0b88679120f94df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.035725 14544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:49.036662 14489 sys_catalog.cc:455] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e33b1af2ed14571b0b88679120f94df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.036772 14489 sys_catalog.cc:458] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:49.037062 14548 catalog_manager.cc:1347] Generated new cluster ID: 984ac5b4b0da43f1b94f2866f781a325
I20260809 23:13:49.037130 14548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 23:13:49.038095 14515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2e33b1af2ed14571b0b88679120f94df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.038291 14515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df [sys.catalog]: This master's current role is: LEADER
I20260809 23:13:49.038931 14467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2e33b1af2ed14571b0b88679120f94df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e33b1af2ed14571b0b88679120f94df" member_type: VOTER last_known_addr { host: "127.8.141.254" port: 38349 } } peers { permanent_uuid: "da4a111058f740dda534844438e25abf" member_type: VOTER last_known_addr { host: "127.8.141.253" port: 40013 } } peers { permanent_uuid: "7be0cd7a4dea4a029c8aadaf2875b9d7" member_type: VOTER last_known_addr { host: "127.8.141.252" port: 37719 } } }
I20260809 23:13:49.039064 14467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 23:13:49.045396 14551 mvcc.cc:205] Tried to move back new op lower bound from 7316755370100125696 to 7316755370047467520. Current Snapshot: MvccSnapshot[applied={T|T < 7316755370100125696}]
I20260809 23:13:49.087000 14548 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 23:13:49.087801 14548 catalog_manager.cc:1504] Loading token signing keys...
I20260809 23:13:49.092998 14548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e33b1af2ed14571b0b88679120f94df: Generated new TSK 0
I20260809 23:13:49.093230 14548 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 23:13:49.123201 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.121563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.8.141.253:40013)
0809 23:13:49.121606 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.121608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.122441 (+ 833us) server_negotiation.cc:407] Connection header received
0809 23:13:49.122556 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.122557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.122571 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.122604 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.123119 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54774: BlockingRecv error: recv got EOF from 127.0.0.1:54774 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 23:13:49.124109 14223 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.121270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60580 (local address 127.8.141.254:38349)
0809 23:13:49.121327 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.121329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.123455 (+ 2126us) server_negotiation.cc:407] Connection header received
0809 23:13:49.123559 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.123560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.123572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.123614 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.124037 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60580: BlockingRecv error: recv got EOF from 127.0.0.1:60580 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 23:13:49.124883 14464 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.122010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48180 (local address 127.8.141.252:37719)
0809 23:13:49.122053 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.122055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.124371 (+ 2316us) server_negotiation.cc:407] Connection header received
0809 23:13:49.124469 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.124470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.124481 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.124511 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.124821 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48180: BlockingRecv error: recv got EOF from 127.0.0.1:48180 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 23:13:49.127506 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.126619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54784 (local address 127.8.141.253:40013)
0809 23:13:49.126663 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.126665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.126904 (+ 239us) server_negotiation.cc:407] Connection header received
0809 23:13:49.127019 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.127020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.127042 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.127073 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.127453 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54784: BlockingRecv error: recv got EOF from 127.0.0.1:54784 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 23:13:49.130872 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.129963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.8.141.253:40013)
0809 23:13:49.130001 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.130003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.130297 (+ 294us) server_negotiation.cc:407] Connection header received
0809 23:13:49.130402 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.130404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.130415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.130445 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.130811 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54792: BlockingRecv error: recv got EOF from 127.0.0.1:54792 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:13:49.135105 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.134186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54798 (local address 127.8.141.253:40013)
0809 23:13:49.134228 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.134230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.134503 (+ 273us) server_negotiation.cc:407] Connection header received
0809 23:13:49.134608 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.134610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.134621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.134653 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.135051 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54798: BlockingRecv error: recv got EOF from 127.0.0.1:54798 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 23:13:49.141526 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.140509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.8.141.253:40013)
0809 23:13:49.140553 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.140555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.140824 (+ 269us) server_negotiation.cc:407] Connection header received
0809 23:13:49.140926 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.140927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.140939 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.140970 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.141463 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54800: BlockingRecv error: recv got EOF from 127.0.0.1:54800 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 23:13:49.151836 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.150918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.8.141.253:40013)
0809 23:13:49.150965 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.150967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.151244 (+ 277us) server_negotiation.cc:407] Connection header received
0809 23:13:49.151345 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.151345 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.151358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.151389 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.151772 (+ 383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54812: BlockingRecv error: recv got EOF from 127.0.0.1:54812 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 23:13:49.170203 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.169373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54828 (local address 127.8.141.253:40013)
0809 23:13:49.169427 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.169429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.169554 (+ 125us) server_negotiation.cc:407] Connection header received
0809 23:13:49.169641 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.169642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.169658 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.169735 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.170124 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54828: BlockingRecv error: recv got EOF from 127.0.0.1:54828 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 23:13:49.204375 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.203566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54832 (local address 127.8.141.253:40013)
0809 23:13:49.203626 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.203628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.203737 (+ 109us) server_negotiation.cc:407] Connection header received
0809 23:13:49.203819 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.203821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.203834 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.203893 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.204302 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54832: BlockingRecv error: recv got EOF from 127.0.0.1:54832 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 23:13:49.270548 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.269737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.8.141.253:40013)
0809 23:13:49.269838 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.269840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.269957 (+ 117us) server_negotiation.cc:407] Connection header received
0809 23:13:49.270007 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.270008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.270020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.270073 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.270496 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54842: BlockingRecv error: recv got EOF from 127.0.0.1:54842 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 23:13:49.400720 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.400023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54854 (local address 127.8.141.253:40013)
0809 23:13:49.400100 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.400101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.400111 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:13:49.400140 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.400141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.400151 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.400226 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.400670 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54854: BlockingRecv error: recv got EOF from 127.0.0.1:54854 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 23:13:49.659286 14342 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 23:13:49.658383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54858 (local address 127.8.141.253:40013)
0809 23:13:49.658475 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:49.658477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:49.658594 (+ 117us) server_negotiation.cc:407] Connection header received
0809 23:13:49.658628 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:49.658629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:49.658642 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:49.658695 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 23:13:49.659206 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54858: BlockingRecv error: recv got EOF from 127.0.0.1:54858 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 23:13:49.940811 14485 catalog_manager.cc:1259] Loaded cluster ID: 984ac5b4b0da43f1b94f2866f781a325
I20260809 23:13:49.940932 14485 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: loading cluster ID for follower catalog manager: success
I20260809 23:13:49.942138 14485 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: acquiring CA information for follower catalog manager: success
I20260809 23:13:49.942559 14485 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7be0cd7a4dea4a029c8aadaf2875b9d7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:13:49.986018 14510 catalog_manager.cc:1259] Loaded cluster ID: 984ac5b4b0da43f1b94f2866f781a325
I20260809 23:13:49.986107 14510 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: loading cluster ID for follower catalog manager: success
I20260809 23:13:49.986974 14510 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: acquiring CA information for follower catalog manager: success
I20260809 23:13:49.987304 14510 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da4a111058f740dda534844438e25abf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 23:13:50.175450 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:50.172735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54872 (local address 127.8.141.253:40013)
0809 23:13:50.172877 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:50.172879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:50.172891 (+ 12us) server_negotiation.cc:407] Connection header received
0809 23:13:50.172954 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:50.172955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:50.173022 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:50.173116 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:50.173280 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.173827 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.174610 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.174824 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.174904 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:50.175087 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:50.175195 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:50.175288 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:50.175312 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 23:13:50.175334 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":58,"threads_started":1}
I20260809 23:13:50.179615 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:50.177304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.8.141.252:37719)
0809 23:13:50.177461 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:50.177463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:50.177473 (+ 10us) server_negotiation.cc:407] Connection header received
0809 23:13:50.177548 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:50.177549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:50.177585 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:50.177657 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:50.177729 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.178220 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.178886 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.179081 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.179137 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:50.179277 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:50.179373 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:50.179431 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:50.179474 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 23:13:50.179492 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260809 23:13:50.185034 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:50.181539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54888 (local address 127.8.141.253:40013)
0809 23:13:50.181591 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:50.181594 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:50.181824 (+ 230us) server_negotiation.cc:407] Connection header received
0809 23:13:50.181904 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:50.181906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:50.181918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:50.181983 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:50.182133 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.182939 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.184099 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.184477 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.184549 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:50.184712 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:50.184756 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:50.184830 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:50.184916 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 23:13:50.184944 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 23:13:50.185473 14690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:50.181739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.8.141.252:37719)
0809 23:13:50.181799 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:50.181801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:50.182061 (+ 260us) server_negotiation.cc:407] Connection header received
0809 23:13:50.182089 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:50.182090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:50.182104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:50.182155 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:50.182357 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.183099 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.184227 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.184560 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.184626 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:50.184822 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:50.184855 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:50.184908 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:50.185371 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 23:13:50.185399 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 23:13:50.185762 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 23:13:50.181480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.8.141.254:38349)
0809 23:13:50.181649 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 23:13:50.181651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 23:13:50.181791 (+ 140us) server_negotiation.cc:407] Connection header received
0809 23:13:50.181888 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 23:13:50.181889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 23:13:50.181945 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 23:13:50.182023 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 23:13:50.182259 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.183036 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.184199 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 23:13:50.184541 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 23:13:50.184607 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 23:13:50.184885 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 23:13:50.185000 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 23:13:50.185081 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 23:13:50.185522 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 23:13:50.185659 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260809 23:13:50.187296 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 14089
I20260809 23:13:50.209616 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 14227
I20260809 23:13:50.220691 8759 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task1V3VFd/build/release/bin/kudu with pid 14349
2026-08-09T23:13:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3295 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6620 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (304068 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST